-- Logs begin at Thu 2022-06-16 13:59:27 UTC, end at Thu 2022-06-16 15:13:46 UTC. -- Jun 16 14:49:26.346842 dv-cinder-837353-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 14:49:26.355225 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 14:49:26 2022] *** Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: nodename: dv-cinder-837353-2-iscsi Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: machine: x86_64 Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: clock source: unix Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: pcre jit disabled Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: detected number of CPU cores: 4 Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: current working directory: / Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: detected binary path: /usr/bin/uwsgi-core Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: your processes number limit is 47680 Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: your memory page size is 4096 bytes Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: detected max file descriptor number: 2048 Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: lock engine: pthread robust mutexes Jun 16 14:49:26.358590 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: thunder lock: enabled Jun 16 14:49:26.359388 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 14:49:26.359388 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 14:49:26.405228 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: Python main interpreter initialized at 0x56381dc599c0 Jun 16 14:49:26.405228 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: python threads support enabled Jun 16 14:49:26.405228 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: your server socket listen backlog is limited to 100 connections Jun 16 14:49:26.405228 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: your mercy for graceful operations on workers is 90 seconds Jun 16 14:49:26.405618 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 14:49:26.405699 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: *** Operational MODE: preforking *** Jun 16 14:49:26.405951 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: *** uWSGI is running in multiple interpreter mode *** Jun 16 14:49:26.405951 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: spawned uWSGI master process (pid: 80406) Jun 16 14:49:26.406162 dv-cinder-837353-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 14:49:26.407045 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: spawned uWSGI worker 1 (pid: 80419, cores: 1) Jun 16 14:49:26.407151 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: spawned uWSGI worker 2 (pid: 80420, cores: 1) Jun 16 14:49:26.407329 dv-cinder-837353-2-iscsi devstack@keystone.service[80406]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 14:49:32.107430 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 14:49:32.107430 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: from cryptography.utils import int_from_bytes Jun 16 14:49:32.108476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 14:49:32.108476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: from cryptography.utils import int_from_bytes Jun 16 14:49:32.159809 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 14:49:32.160081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 14:49:32.160205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] command line args: [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 14:49:32.160511 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 14:49:32.160700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ================================================================================ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 14:49:32.161880 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] admin_token = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.161880 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] config_dir = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.161880 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.161880 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] config_source = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.161880 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] debug = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.162389 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.162389 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.162778 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.162778 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.163019 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.163258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.163451 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_config_append = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.163636 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.163820 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_dir = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.164005 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_file = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.164267 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.164528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.164789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.165042 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.165255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.165974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.166236 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.166484 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.166729 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.166978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.167516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.167516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.167739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.167942 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.168240 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.168774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.168961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] publish_errors = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.169202 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.169441 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.169623 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.169877 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.170125 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.170370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] standard_threads = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.170607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.170697 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 14:49:32.170697 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: from cryptography.utils import int_from_bytes Jun 16 14:49:32.171382 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.171382 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.171382 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_journal = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.171601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 14:49:32.171601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: from cryptography.utils import int_from_bytes Jun 16 14:49:32.171772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_json = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.171772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_stderr = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.171772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] use_syslog = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.171987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.172175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.172448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.172718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.172967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.173223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.173476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.173734 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.174134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.174392 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.174650 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.174912 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.175163 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.175415 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.175661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.175915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.176170 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.176443 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.176707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.176971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.177226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.177470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.177731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.178088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.178266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.178524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.178762 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.179000 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.179237 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.179549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.179740 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.179995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.180249 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.180493 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.180749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.181041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.181321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.181561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.181781 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.182580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.182833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.183256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.183365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.184036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.184303 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.184555 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.184816 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.185060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.185306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.185546 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.185790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.186034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.186289 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.186529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.186779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.187022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.187271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.187512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.187757 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.187995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.188285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.188519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.188765 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.189006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.189275 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.189519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.189759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.190000 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.190242 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.190521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.190769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.191025 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.191282 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.191536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.191769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.192007 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.192282 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.192529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.192771 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.193021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.193259 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.193501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.193740 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.193985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.194224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.194493 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.194740 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.194985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.195818 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.195818 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.195818 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.198973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.198973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.198973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.198973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.198973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.198973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.198973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.198973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.198973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.200331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.200331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.200331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.200331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.200331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.200331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.external = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.200331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.200331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.200331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.201370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.password = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.201370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] auth.token = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.201370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.201370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.201370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.201370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.202021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.202021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.202167 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.202731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.202731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.202964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.203186 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.203430 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.203679 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.203925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.204185 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.204472 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.204716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.204976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.205274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.205525 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.205795 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.206044 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.206306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.206548 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.206791 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.207160 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.207295 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.207533 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.207786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.208032 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.208319 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.208574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.208832 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.209079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.209329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.209571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.209825 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.210080 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.210329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.210568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.210821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.211058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.211303 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.211557 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.211817 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.212057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.212866 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.213079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.213332 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.213587 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.213843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.214716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.214716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.214716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.214938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.215073 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.215316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.215557 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.215795 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.216036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.216316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.216562 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.216819 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.217057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.217310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.217560 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.217801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.218052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.218294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.218529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.218773 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.219009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.219253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.219496 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.219748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.219990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.220311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.220714 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.220790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.221012 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.221261 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.221438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.222014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.222014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.222224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.222373 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.222608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.222853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.223117 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.223353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.223605 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.223857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.224113 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.224378 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.224616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.224855 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.225095 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.225336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.225571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.225823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.226074 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.226326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.226577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.226827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.227061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.227320 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.227554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.227887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.228707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.228883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.229131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.229390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.229748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.230014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.230255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.230510 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.230757 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.230997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.231241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.231502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.231884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.232131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.232405 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.232657 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.232916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.233176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.233419 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.233672 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.233951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.234228 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.234415 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.234656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.234900 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.235141 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] role.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.235393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] role.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.235633 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.235886 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.236143 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.236420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.236681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.236933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.237390 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 14:49:32.237530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.237655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 14:49:32.237785 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.237913 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] command line args: [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 14:49:32.238060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.238192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.238274 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 14:49:32.238375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.238458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ================================================================================ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 14:49:32.238628 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.238795 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] admin_token = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.238884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.239091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] config_dir = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.239184 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.239287 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.239477 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] config_source = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.239564 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.239665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] debug = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.239907 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.239986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.240085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.240161 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.240313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.240392 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.240521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.240605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.240777 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.240859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.241019 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.241109 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.241186 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_config_append = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.241282 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.241370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.241542 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.241670 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_dir = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.241786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.241869 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_file = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.242023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.242106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.242262 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.242342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.242499 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.242582 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.242756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.242837 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.242964 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.243044 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.243232 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.243316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.243651 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.243848 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.244031 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.244301 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.244522 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.244607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.244775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.244856 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.245033 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.245263 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.245354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.245524 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.245595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.245766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.245848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.246010 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.246100 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.246263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.246463 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.246535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.246756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.247001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.247091 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.247244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.247577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.247729 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.247827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.247930 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] publish_errors = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.248108 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.248198 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.248281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.248463 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.248680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.248761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.249017 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.249097 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.249268 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.249434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.249516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.249753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.249834 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.250003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.250227 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] standard_threads = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.250308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.250478 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.250634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.250716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.250953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.251045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.251204 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.251353 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.251443 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.251670 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_journal = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.251751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.251918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.252050 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_json = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.252182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.252366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_stderr = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.252447 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.252605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] use_syslog = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.252696 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.252888 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.253027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:49:32.253137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.253379 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.253486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.253567 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 14:49:32.253902 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.254350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.254763 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.255205 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.255634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.256071 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.256555 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.256945 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.257274 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.257552 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.257805 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.258071 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.258324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.258588 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.258850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.259282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.260397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.260683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.260946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.261202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.261629 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.261814 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.261999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.262256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.262496 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.262861 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.263104 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.263355 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.263748 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.264007 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.264138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.264393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.264652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.264907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.265152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.265407 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.265643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.265967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.266219 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.266454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.266706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.266921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.267156 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.267397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.267650 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.267886 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.268125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.268386 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.268626 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.270161 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.270437 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.270686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.271006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.271455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.271455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.271673 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.271916 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.272161 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.272456 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.272686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.272929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.273866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.273866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.273866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.273866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.274375 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.274375 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.275140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.275559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.276865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.276865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.276865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.277072 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.277259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.277515 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.277776 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.278000 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.278282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.278489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.278730 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.279943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.279943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.279943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.279943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.279943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.280362 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.280433 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.280634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.280873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.281110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.281796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.281796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.281796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.282713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.282713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.282713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.282713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.282922 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.283209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.283644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.283715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.283948 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.284412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.284848 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.external = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.285289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.285828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.286145 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.286970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.password = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.286970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] auth.token = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.286970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.287285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.288128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.288128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.288310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.288576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.289908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.289908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.289908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.289908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.289908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.290190 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.291049 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.292478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.293531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.293531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.293531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.293531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.293980 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.294431 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.294851 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.295045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.295316 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.295565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.295811 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.296064 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.296357 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.296594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.296971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.297212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.297474 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.297713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.297972 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.298214 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.298557 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.298806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.299239 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.299683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.300124 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.300589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.301088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.301315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.301663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.302040 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.302321 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.302562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.302944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.303228 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.303488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.303775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.304218 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.304530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.305397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.305397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.305397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.305688 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.306272 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.308300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.308520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.308777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.309039 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.309320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.309589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.309852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.310223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.310223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.310223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.311173 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.311173 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.311321 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.311565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.311799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.312041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.312308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.312564 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.312806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.313047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.313304 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.313691 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.313931 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.314166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.314421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.314674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.314915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.315152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.315388 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.315621 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.315865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.316129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.316388 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.316672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.316935 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.317272 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.317534 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.317800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.318040 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.318279 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.318519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.318754 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.319008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.319256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.319510 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.319757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.320006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.320275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.320508 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.320749 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.320985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.321223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.321487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.321861 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.322759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.323010 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.323444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.323747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.324528 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.324767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.325161 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.325593 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.326062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.326511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.326947 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.327387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.327831 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.328326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.328746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.329186 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.329636 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.330086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.330499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.330874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.331287 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.331539 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.332542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] role.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.332542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] role.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.332542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.333480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.333637 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.333899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.334158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.334445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.334704 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.334960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.335217 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.335614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.335972 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.336468 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.336933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.337857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.338818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.339069 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.339330 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.339590 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.339848 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.340108 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.340379 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.340647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.340904 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.341148 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.341385 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.341622 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.341857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.342110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.342368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.342620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.342860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.343099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.343331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.343567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.343805 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.344058 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.344322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.344693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.345053 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.345203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.345451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.345692 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.345980 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.346224 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.346465 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.346702 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.346950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.347203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.347435 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.347683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.348088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.348383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.348630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.348880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.349120 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.349366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.349602 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.349838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.350073 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.350310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.351077 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.351318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.351570 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.351812 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.352152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.352492 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.353446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.354157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.354408 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.354744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:49:32.354841 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80419) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 14:49:32.376554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.376744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.376915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.388141 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.419644 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.419856 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.479726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.488282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.488517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.488743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.498085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.522635 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.522867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.536644 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.536887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.579403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.616422 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.616696 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.616869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.617093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.617315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.617536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.636530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.636706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.642985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.643231 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.643489 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.643787 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.644055 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.644310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.644536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.656894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.670616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.670897 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.671082 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.671315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.671536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.671747 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.671966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.690494 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.690747 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 14:49:32.691041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.691254 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 14:49:32.691651 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.692059 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.692330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.692504 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.704825 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.705196 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.705388 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.705680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.713923 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.714297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.714663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.714955 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.715061 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.715368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.715733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.718313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.718596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.718884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.719169 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.719452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.719714 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.719990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.720295 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.720572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.720849 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.721122 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.735036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.735268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.735473 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.735692 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.736679 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.736909 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.737127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.749250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.750311 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.750915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.751202 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.754847 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.755197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.755563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.755946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.756376 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.756643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.756818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.756959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80419) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:49:32.757093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.757516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.765903 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.766201 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.769186 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.769466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 14:49:32.769872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.770143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 14:49:32.770488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.770755 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.770875 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.771046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.771343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.771443 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.771632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.771884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.771992 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.772172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.772470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.772748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.789691 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.789984 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.790327 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.790658 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.791705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.792127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.797559 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.800444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.800726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.800867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.807650 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.808816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.809321 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.809779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.810122 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.810470 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.810863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.811213 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.811530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.811619 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.811918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.812285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.812635 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.815264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.815565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.815850 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.816134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.816441 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.816726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.836208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.836529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.836802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.837602 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.845702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.847441 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.847725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.850329 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.852384 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.852629 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.852903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.853551 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.853789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.854146 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.854431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.860152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.860472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.860757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.868354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.868660 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.868976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.869366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.869793 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.870080 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.870383 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.872341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.878421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.878714 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.878998 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.879275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.879542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.879833 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.880128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.880720 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.889900 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.893198 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.896578 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.898395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.900656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.901019 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.901460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.909120 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.909415 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.909717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.910917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.914229 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.914522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.914939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.915319 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.915909 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.919914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.920212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.925229 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.925543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.925833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.927633 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.932057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.934017 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.934310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.934595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.934876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.935164 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.935561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.935846 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.936127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.936461 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.936739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.936955 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.937054 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.937260 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.937361 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.937543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.937648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.937803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.937886 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.938084 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.938448 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.967205 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.967501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.967818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.968205 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.976468 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56381dc599c0 pid: 80420 (default app) Jun 16 14:49:32.978523 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.980977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.981110 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543620-57b2-4a2b-bec5-cf8c8bd214f8 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:32.981281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543620-57b2-4a2b-bec5-cf8c8bd214f8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:32.981371 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.981569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543620-57b2-4a2b-bec5-cf8c8bd214f8 None None] PATH_INFO: `/v3` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:32.985898 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 1/1] 10.222.0.88 () {50 vars in 791 bytes} [Thu Jun 16 14:49:32 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:32.986615 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:32.986615 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.986979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.987182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:32.987472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.002896 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:33.003220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.003501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.003784 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.004065 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.004651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.005163 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.024402 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:33.028532 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:33.031925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.033614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.035450 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:33.035751 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.036056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.044288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.045952 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:33.049223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.049502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.049773 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.050057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.061618 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:33.061913 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.062197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.070176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80419) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:49:33.070457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.070784 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.071120 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.071279 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.071636 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.071910 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.072184 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.072582 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.072851 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.073132 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.073403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.073681 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.073955 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80419) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:49:33.113946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56381dc599c0 pid: 80419 (default app) Jun 16 14:49:42.364569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a38c4-a6d9-4479-9575-5568400e65cd None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:42.365123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a38c4-a6d9-4479-9575-5568400e65cd None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:42.365211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a38c4-a6d9-4479-9575-5568400e65cd None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:42.366849 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 2/2] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:49:42.377073 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:42.377908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:42.377908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:42.448939 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: INFO dbcounter [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] Registered counter for database keystone Jun 16 14:49:42.471924 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG oslo_db.sqlalchemy.engines [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80419) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 14:49:42.479460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writer thread running {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 14:49:42.683347 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG passlib.handlers.bcrypt [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80419) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 14:49:42.683697 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG passlib.handlers.bcrypt [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80419) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 14:49:42.741936 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:42.744542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:42.879437 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 1/3] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 16 14:49:43.078066 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a38c4-a6d9-4479-9575-5568400e65cd None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:43.150878 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: INFO dbcounter [None req-7f9a38c4-a6d9-4479-9575-5568400e65cd None None] Registered counter for database keystone Jun 16 14:49:43.171091 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG oslo_db.sqlalchemy.engines [None req-7f9a38c4-a6d9-4479-9575-5568400e65cd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80420) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 14:49:43.177854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writer thread running {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 14:49:43.303550 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a38c4-a6d9-4479-9575-5568400e65cd None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:43.308517 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6d8d-e015-4b34-8e6e-12e5928d2695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:43.309224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6d8d-e015-4b34-8e6e-12e5928d2695 admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:43.309515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6d8d-e015-4b34-8e6e-12e5928d2695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:43.309807 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6d8d-e015-4b34-8e6e-12e5928d2695 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:43.319071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b6d8d-e015-4b34-8e6e-12e5928d2695 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:43.384329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fb9b6d8d-e015-4b34-8e6e-12e5928d2695 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 14:49:43.385220 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b6d8d-e015-4b34-8e6e-12e5928d2695 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:43.395435 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 3/4] 10.222.0.88 () {58 vars in 1324 bytes} [Thu Jun 16 14:49:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 14:49:43.402273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:43.420290 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87f0f9-f598-4f7b-9ed0-041e4a094f01 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:43.424772 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8db6fc-a4ec-4504-89f5-8ac658ec1004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:43.425930 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8db6fc-a4ec-4504-89f5-8ac658ec1004 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:43.426222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8db6fc-a4ec-4504-89f5-8ac658ec1004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:43.426531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8db6fc-a4ec-4504-89f5-8ac658ec1004 admin admin] PATH_INFO: `/v3/projects/7d718157b3b949fc80592326cc9cfb5a` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:43.428057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8db6fc-a4ec-4504-89f5-8ac658ec1004 admin admin] RBAC: Authorizing `identity:get_project(project_id=7d718157b3b949fc80592326cc9cfb5a)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:43.479272 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1f8db6fc-a4ec-4504-89f5-8ac658ec1004 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 14:49:43.479616 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8db6fc-a4ec-4504-89f5-8ac658ec1004 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:43.484392 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 2/5] 10.222.0.88 () {56 vars in 1207 bytes} [Thu Jun 16 14:49:43 2022] GET /identity/v3/projects/7d718157b3b949fc80592326cc9cfb5a => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:45.788825 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7d174-af49-4ecc-ad17-0478b0422458 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:45.789453 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7d174-af49-4ecc-ad17-0478b0422458 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:45.789453 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7d174-af49-4ecc-ad17-0478b0422458 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:45.794336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 4/6] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:49:45.800383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30095d-60b7-46a8-af59-be900f2c8a6d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:45.801095 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30095d-60b7-46a8-af59-be900f2c8a6d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:45.801095 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30095d-60b7-46a8-af59-be900f2c8a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:45.823461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-dd30095d-60b7-46a8-af59-be900f2c8a6d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:45.825049 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-dd30095d-60b7-46a8-af59-be900f2c8a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:45.833296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 3/7] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 14:49:45.857184 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7d174-af49-4ecc-ad17-0478b0422458 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:45.870506 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7d174-af49-4ecc-ad17-0478b0422458 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:45.875133 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8daeb9-4176-421e-ae2b-c3c9db8d731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:45.875776 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8daeb9-4176-421e-ae2b-c3c9db8d731b admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:45.876065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8daeb9-4176-421e-ae2b-c3c9db8d731b admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:45.876454 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8daeb9-4176-421e-ae2b-c3c9db8d731b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:45.887073 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8daeb9-4176-421e-ae2b-c3c9db8d731b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:45.889209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8daeb9-4176-421e-ae2b-c3c9db8d731b admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:45.900159 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 5/8] 10.222.0.88 () {58 vars in 1324 bytes} [Thu Jun 16 14:49:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 14:49:45.905926 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30095d-60b7-46a8-af59-be900f2c8a6d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:45.916137 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30095d-60b7-46a8-af59-be900f2c8a6d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:45.920717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515a64b-40d3-4c80-b8e2-8ce5dc4caddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:45.921600 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515a64b-40d3-4c80-b8e2-8ce5dc4caddc admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:45.921892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515a64b-40d3-4c80-b8e2-8ce5dc4caddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:45.922177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515a64b-40d3-4c80-b8e2-8ce5dc4caddc admin admin] PATH_INFO: `/v3/users/b841e83eb2b74ca1b301a4644bbdbe93` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:45.923605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515a64b-40d3-4c80-b8e2-8ce5dc4caddc admin admin] RBAC: Authorizing `identity:get_user(user_id=b841e83eb2b74ca1b301a4644bbdbe93)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:45.925517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515a64b-40d3-4c80-b8e2-8ce5dc4caddc admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:45.927166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 4/9] 10.222.0.88 () {56 vars in 1198 bytes} [Thu Jun 16 14:49:45 2022] GET /identity/v3/users/b841e83eb2b74ca1b301a4644bbdbe93 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:48.187111 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202109d8-0dda-4c6b-8df5-dc1c0353ff24 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:48.187908 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202109d8-0dda-4c6b-8df5-dc1c0353ff24 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:48.187908 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202109d8-0dda-4c6b-8df5-dc1c0353ff24 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:48.189386 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 6/10] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:49:48.197676 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ab7cf-fbf3-405d-9caa-3fca9cbc0ae5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:48.198201 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ab7cf-fbf3-405d-9caa-3fca9cbc0ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:48.198285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ab7cf-fbf3-405d-9caa-3fca9cbc0ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:48.220835 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d37ab7cf-fbf3-405d-9caa-3fca9cbc0ae5 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:48.222099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d37ab7cf-fbf3-405d-9caa-3fca9cbc0ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:48.281402 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccc47d-eece-4c39-af23-aae5b8add1d2 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:48.281758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccc47d-eece-4c39-af23-aae5b8add1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:48.282105 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccc47d-eece-4c39-af23-aae5b8add1d2 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:48.284520 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 7/11] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:48.294038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8addc2-7cd6-47c6-9263-eaec9d051993 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:48.294333 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8addc2-7cd6-47c6-9263-eaec9d051993 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:48.294621 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8addc2-7cd6-47c6-9263-eaec9d051993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:48.324100 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 5/12] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:49:48.335665 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG passlib.handlers.bcrypt [None req-9d8addc2-7cd6-47c6-9263-eaec9d051993 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80420) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 14:49:48.336096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG passlib.handlers.bcrypt [None req-9d8addc2-7cd6-47c6-9263-eaec9d051993 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80420) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 14:49:48.347543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ab7cf-fbf3-405d-9caa-3fca9cbc0ae5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:48.372358 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:48.382460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9d8addc2-7cd6-47c6-9263-eaec9d051993 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:48.385104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9d8addc2-7cd6-47c6-9263-eaec9d051993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:48.461623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ab7cf-fbf3-405d-9caa-3fca9cbc0ae5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:48.496143 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 8/13] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:49:48.515653 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e366b-e3c5-4d6f-86a2-12b0f4395cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:48.519178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e366b-e3c5-4d6f-86a2-12b0f4395cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:48.519178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e366b-e3c5-4d6f-86a2-12b0f4395cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:48.519178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e366b-e3c5-4d6f-86a2-12b0f4395cf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:48.520725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8addc2-7cd6-47c6-9263-eaec9d051993 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:48.530715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e366b-e3c5-4d6f-86a2-12b0f4395cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3c0e366b-e3c5-4d6f-86a2-12b0f4395cf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:49:48.542266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:49:48.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:49:48.544389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 6/14] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:49:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:49:48.549506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e366b-e3c5-4d6f-86a2-12b0f4395cf6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:48.628875 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8addc2-7cd6-47c6-9263-eaec9d051993 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:48.631210 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:48.637504 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:48.657793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e366b-e3c5-4d6f-86a2-12b0f4395cf6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:48.669400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bf11e-d10b-4826-b680-b263cc1d486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:48.670400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bf11e-d10b-4826-b680-b263cc1d486d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:48.670697 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bf11e-d10b-4826-b680-b263cc1d486d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:48.670982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bf11e-d10b-4826-b680-b263cc1d486d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:48.708746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff31467-f61a-40e9-b74f-a9347217a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:48.709807 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff31467-f61a-40e9-b74f-a9347217a391 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:48.710316 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff31467-f61a-40e9-b74f-a9347217a391 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:48.711196 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff31467-f61a-40e9-b74f-a9347217a391 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:48.711960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff31467-f61a-40e9-b74f-a9347217a391 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:49:48.712475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff31467-f61a-40e9-b74f-a9347217a391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:48.714845 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff31467-f61a-40e9-b74f-a9347217a391 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:48.730330 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 7/15] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:49:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:49:48.738769 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff31467-f61a-40e9-b74f-a9347217a391 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:48.758032 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bf11e-d10b-4826-b680-b263cc1d486d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:48.760417 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bf11e-d10b-4826-b680-b263cc1d486d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:48.848353 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff31467-f61a-40e9-b74f-a9347217a391 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:48.871565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 9/16] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:49:48.877324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bf11e-d10b-4826-b680-b263cc1d486d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:48.880163 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:48.884836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102682f3-89ba-4193-b8d9-ab04cb5a3fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:48.885714 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102682f3-89ba-4193-b8d9-ab04cb5a3fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:48.886001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102682f3-89ba-4193-b8d9-ab04cb5a3fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:48.886337 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102682f3-89ba-4193-b8d9-ab04cb5a3fd7 None admin] PATH_INFO: `/v3/users/b841e83eb2b74ca1b301a4644bbdbe93` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:48.887648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102682f3-89ba-4193-b8d9-ab04cb5a3fd7 None admin] RBAC: Authorizing `identity:get_user(user_id=b841e83eb2b74ca1b301a4644bbdbe93)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:48.889204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102682f3-89ba-4193-b8d9-ab04cb5a3fd7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:48.899432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 8/17] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:49:48 2022] GET /identity/v3/users/b841e83eb2b74ca1b301a4644bbdbe93 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:49:48.899432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102682f3-89ba-4193-b8d9-ab04cb5a3fd7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:48.911492 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:48.986812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bf11e-d10b-4826-b680-b263cc1d486d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:49.011517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102682f3-89ba-4193-b8d9-ab04cb5a3fd7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:49.030709 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7d63-e094-4a89-aef1-cdc17fe43130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:49.031518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7d63-e094-4a89-aef1-cdc17fe43130 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:49.031803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7d63-e094-4a89-aef1-cdc17fe43130 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:49.032086 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba7d63-e094-4a89-aef1-cdc17fe43130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:49.053905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba7d63-e094-4a89-aef1-cdc17fe43130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:49.056103 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c3d7-4224-4956-97af-64196f8ec162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:49.056337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba7d63-e094-4a89-aef1-cdc17fe43130 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:49.056952 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c3d7-4224-4956-97af-64196f8ec162 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:49.057241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c3d7-4224-4956-97af-64196f8ec162 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:49.057543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c3d7-4224-4956-97af-64196f8ec162 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:49.058772 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1c3d7-4224-4956-97af-64196f8ec162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:49.060264 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1c3d7-4224-4956-97af-64196f8ec162 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:49.061905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 9/18] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:49:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:49:49.067741 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c3d7-4224-4956-97af-64196f8ec162 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1dba7d63-e094-4a89-aef1-cdc17fe43130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:49:49.077401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:49:49.078255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:49:49.079071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:49:49.079071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:49:49.079071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:49:49.079071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:49:49.079071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 10/19] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:49:49.084985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7d63-e094-4a89-aef1-cdc17fe43130 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:49.150599 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:49.157974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:49.184717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c3d7-4224-4956-97af-64196f8ec162 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:49.205359 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba7d63-e094-4a89-aef1-cdc17fe43130 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:49.231193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838594e-21a3-4db4-ae3f-0b6c87ccd483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:49.232081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838594e-21a3-4db4-ae3f-0b6c87ccd483 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:49.232599 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838594e-21a3-4db4-ae3f-0b6c87ccd483 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:49.233084 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838594e-21a3-4db4-ae3f-0b6c87ccd483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:49.234417 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838594e-21a3-4db4-ae3f-0b6c87ccd483 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.domain.id=default, user.id=b841e83eb2b74ca1b301a4644bbdbe93) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:49:49.234923 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838594e-21a3-4db4-ae3f-0b6c87ccd483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:49.237052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838594e-21a3-4db4-ae3f-0b6c87ccd483 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:49.245099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df332d-0b2d-4086-8cc6-2bbdc5d3f669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:49.245782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df332d-0b2d-4086-8cc6-2bbdc5d3f669 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:49.246143 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df332d-0b2d-4086-8cc6-2bbdc5d3f669 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:49.246431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df332d-0b2d-4086-8cc6-2bbdc5d3f669 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:49.246908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 10/20] 10.222.0.88 () {56 vars in 1322 bytes} [Thu Jun 16 14:49:49 2022] GET /identity/v3/role_assignments?user.id=b841e83eb2b74ca1b301a4644bbdbe93&scope.domain.id=default&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 233 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:49:49.247176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df332d-0b2d-4086-8cc6-2bbdc5d3f669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:49:49.247492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df332d-0b2d-4086-8cc6-2bbdc5d3f669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:49.249050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df332d-0b2d-4086-8cc6-2bbdc5d3f669 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:49.271085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 11/21] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:49:52.111113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8ad8d-238f-4eb2-af3e-8488a641241c None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:52.111814 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8ad8d-238f-4eb2-af3e-8488a641241c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:52.112208 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8ad8d-238f-4eb2-af3e-8488a641241c None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:52.114963 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 11/22] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:52.122790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2be6c7-be66-416d-a5c7-91d25048d1cf None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:52.123490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2be6c7-be66-416d-a5c7-91d25048d1cf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:52.123803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2be6c7-be66-416d-a5c7-91d25048d1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:52.158708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ea2be6c7-be66-416d-a5c7-91d25048d1cf None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:52.160416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ea2be6c7-be66-416d-a5c7-91d25048d1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:52.270258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 12/23] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:49:52.339998 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8ad8d-238f-4eb2-af3e-8488a641241c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:52.548070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8ad8d-238f-4eb2-af3e-8488a641241c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:52.611031 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a6074-fbe8-41fe-8f3f-463847e2afee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:52.612308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a6074-fbe8-41fe-8f3f-463847e2afee None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:52.612797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a6074-fbe8-41fe-8f3f-463847e2afee None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:52.613276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a6074-fbe8-41fe-8f3f-463847e2afee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:52.633097 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a6074-fbe8-41fe-8f3f-463847e2afee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3b4a6074-fbe8-41fe-8f3f-463847e2afee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:49:52.649481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:49:52.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:49:52.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:49:52.656408 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 12/24] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:49:52.666358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2be6c7-be66-416d-a5c7-91d25048d1cf None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:52.757578 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:52.830165 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306508e5-b57c-4c28-bd56-40b292dd08eb None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:52.830906 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306508e5-b57c-4c28-bd56-40b292dd08eb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:52.831431 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306508e5-b57c-4c28-bd56-40b292dd08eb None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:52.834339 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 13/25] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:49:52.849090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08949047-e151-4825-8096-d924b3117a11 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:52.849946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08949047-e151-4825-8096-d924b3117a11 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:52.850445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08949047-e151-4825-8096-d924b3117a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:52.851542 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2be6c7-be66-416d-a5c7-91d25048d1cf None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:52.872594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:52.901095 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-08949047-e151-4825-8096-d924b3117a11 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:52.902347 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-08949047-e151-4825-8096-d924b3117a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:52.950036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eda100-e118-42a3-ade4-6c87ce00d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:52.950974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eda100-e118-42a3-ade4-6c87ce00d29e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:52.951173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eda100-e118-42a3-ade4-6c87ce00d29e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:52.951470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eda100-e118-42a3-ade4-6c87ce00d29e None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:52.952282 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eda100-e118-42a3-ade4-6c87ce00d29e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:49:52.952606 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eda100-e118-42a3-ade4-6c87ce00d29e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:52.954247 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eda100-e118-42a3-ade4-6c87ce00d29e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:52.994816 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 13/26] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:53.021684 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eda100-e118-42a3-ade4-6c87ce00d29e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:53.057520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 14/27] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:49:53.084906 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08949047-e151-4825-8096-d924b3117a11 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:53.261708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08949047-e151-4825-8096-d924b3117a11 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:53.301977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff80e2c-2909-4ff3-8be9-904afb0ba276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:53.302869 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff80e2c-2909-4ff3-8be9-904afb0ba276 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:53.303168 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff80e2c-2909-4ff3-8be9-904afb0ba276 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:53.303957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff80e2c-2909-4ff3-8be9-904afb0ba276 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:53.305335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff80e2c-2909-4ff3-8be9-904afb0ba276 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:53.307288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff80e2c-2909-4ff3-8be9-904afb0ba276 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:53.316599 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:53.343347 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eda100-e118-42a3-ade4-6c87ce00d29e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:53.380028 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:53.397180 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5d491-c072-4b91-bdfc-8a70324a28ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:53.398796 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5d491-c072-4b91-bdfc-8a70324a28ef None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:53.399519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5d491-c072-4b91-bdfc-8a70324a28ef None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:53.399999 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5d491-c072-4b91-bdfc-8a70324a28ef None admin] PATH_INFO: `/v3/users/b841e83eb2b74ca1b301a4644bbdbe93` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:53.402954 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5d491-c072-4b91-bdfc-8a70324a28ef None admin] RBAC: Authorizing `identity:get_user(user_id=b841e83eb2b74ca1b301a4644bbdbe93)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:53.405677 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5d491-c072-4b91-bdfc-8a70324a28ef None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:53.409750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 14/28] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:49:53 2022] GET /identity/v3/users/b841e83eb2b74ca1b301a4644bbdbe93 => generated 247 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:53.412205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f61df-69ee-4067-bb85-937cc0ec389a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:53.412681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f61df-69ee-4067-bb85-937cc0ec389a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:53.413281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f61df-69ee-4067-bb85-937cc0ec389a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:53.416744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 15/29] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:53.418156 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f61df-69ee-4067-bb85-937cc0ec389a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:53.592447 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f61df-69ee-4067-bb85-937cc0ec389a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:53.644688 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fa2f6-b70b-4d7a-a9bd-4fca87dbaa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:53.645833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa2f6-b70b-4d7a-a9bd-4fca87dbaa2b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:53.646384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa2f6-b70b-4d7a-a9bd-4fca87dbaa2b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:53.647008 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa2f6-b70b-4d7a-a9bd-4fca87dbaa2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:53.648997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa2f6-b70b-4d7a-a9bd-4fca87dbaa2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:53.650711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa2f6-b70b-4d7a-a9bd-4fca87dbaa2b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:53.653217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 16/30] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:49:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:53.655175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d084c1-0856-47fc-8925-b7c6b72a064a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:53.655851 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d084c1-0856-47fc-8925-b7c6b72a064a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:53.656555 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d084c1-0856-47fc-8925-b7c6b72a064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:53.685315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-02d084c1-0856-47fc-8925-b7c6b72a064a None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:53.687283 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-02d084c1-0856-47fc-8925-b7c6b72a064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:53.776758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:53.778581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 15/31] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 14:49:53 2022] POST /identity/v3/roles => generated 219 bytes in 695 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:49:53.786232 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f96f1-2580-413b-8bf0-0ffc74fc3428 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:53.786715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f96f1-2580-413b-8bf0-0ffc74fc3428 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:53.786877 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f96f1-2580-413b-8bf0-0ffc74fc3428 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:53.791393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 16/32] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:53.792338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f96f1-2580-413b-8bf0-0ffc74fc3428 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:53.814009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 17/33] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:49:53.816635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa34f85-b389-4939-ba9f-ee83595c1e16 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:53.817551 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa34f85-b389-4939-ba9f-ee83595c1e16 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:53.818220 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa34f85-b389-4939-ba9f-ee83595c1e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:53.846227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5fa34f85-b389-4939-ba9f-ee83595c1e16 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:53.847717 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5fa34f85-b389-4939-ba9f-ee83595c1e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:53.986940 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 18/34] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:49:53.988143 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa34f85-b389-4939-ba9f-ee83595c1e16 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:54.034555 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f96f1-2580-413b-8bf0-0ffc74fc3428 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:54.105648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5bc0c-b839-4e76-bdee-eada86290308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:54.107077 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5bc0c-b839-4e76-bdee-eada86290308 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:54.107738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5bc0c-b839-4e76-bdee-eada86290308 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:54.108200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5bc0c-b839-4e76-bdee-eada86290308 None admin] PATH_INFO: `/v3/domains/default/users/b841e83eb2b74ca1b301a4644bbdbe93/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:54.114231 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5bc0c-b839-4e76-bdee-eada86290308 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b841e83eb2b74ca1b301a4644bbdbe93, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:54.116942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5bc0c-b839-4e76-bdee-eada86290308 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:54.229941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 17/35] 10.222.0.88 () {58 vars in 1361 bytes} [Thu Jun 16 14:49:53 2022] PUT /identity/v3/domains/default/users/b841e83eb2b74ca1b301a4644bbdbe93/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:49:54.231504 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5bc0c-b839-4e76-bdee-eada86290308 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:54.232938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa34f85-b389-4939-ba9f-ee83595c1e16 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:54.238089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:54.306062 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584477f1-a869-4183-b536-789bf565cf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:54.307681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584477f1-a869-4183-b536-789bf565cf0c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:54.308394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584477f1-a869-4183-b536-789bf565cf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:54.309070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584477f1-a869-4183-b536-789bf565cf0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:54.328217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584477f1-a869-4183-b536-789bf565cf0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:54.331236 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584477f1-a869-4183-b536-789bf565cf0c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:54.351314 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-584477f1-a869-4183-b536-789bf565cf0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:49:54.369420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:49:54.371448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:49:54.372951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:49:54.372951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:49:54.372951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:49:54.372951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:49:54.376119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 19/36] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:49:54.387821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ef53a-1a4c-4399-a4a3-12e40a893c21 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:54.388699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ef53a-1a4c-4399-a4a3-12e40a893c21 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:54.390485 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ef53a-1a4c-4399-a4a3-12e40a893c21 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:54.393863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 20/37] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:54.403206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11e061-4e28-4091-bf0c-f045e14c8e46 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:54.403917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11e061-4e28-4091-bf0c-f045e14c8e46 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:54.404948 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11e061-4e28-4091-bf0c-f045e14c8e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:54.422074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5bc0c-b839-4e76-bdee-eada86290308 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:54.436377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9c11e061-4e28-4091-bf0c-f045e14c8e46 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:54.438788 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9c11e061-4e28-4091-bf0c-f045e14c8e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:54.494718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682950f-5b2f-412a-b8b3-cf93c855817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:54.495338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682950f-5b2f-412a-b8b3-cf93c855817f None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:54.495626 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682950f-5b2f-412a-b8b3-cf93c855817f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:54.495956 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682950f-5b2f-412a-b8b3-cf93c855817f None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:54.497216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682950f-5b2f-412a-b8b3-cf93c855817f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:54.498318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682950f-5b2f-412a-b8b3-cf93c855817f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:54.567828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 21/38] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:49:54.568975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11e061-4e28-4091-bf0c-f045e14c8e46 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:54.586789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 18/39] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 14:49:54 2022] POST /identity/v3/roles => generated 225 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:49:54.589767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b0c4-3e3a-49a2-95f6-fd4c1dffd64f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:54.590552 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b0c4-3e3a-49a2-95f6-fd4c1dffd64f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:54.591342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b0c4-3e3a-49a2-95f6-fd4c1dffd64f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:54.594230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 19/40] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:54.595927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b0c4-3e3a-49a2-95f6-fd4c1dffd64f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:54.746081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:54.776116 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11e061-4e28-4091-bf0c-f045e14c8e46 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:54.793025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b0c4-3e3a-49a2-95f6-fd4c1dffd64f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:54.819989 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:54.842189 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fdad7-19e3-453f-8475-4c3e9ba17d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:54.842885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fdad7-19e3-453f-8475-4c3e9ba17d8b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:54.843181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fdad7-19e3-453f-8475-4c3e9ba17d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:54.850556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fdad7-19e3-453f-8475-4c3e9ba17d8b None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:54.850556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fdad7-19e3-453f-8475-4c3e9ba17d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:49:54.850556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fdad7-19e3-453f-8475-4c3e9ba17d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:54.850556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fdad7-19e3-453f-8475-4c3e9ba17d8b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:54.857609 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260de92-3a57-48e3-b899-f3e5cd515cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:54.859229 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260de92-3a57-48e3-b899-f3e5cd515cca None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:54.860040 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260de92-3a57-48e3-b899-f3e5cd515cca None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:54.860550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260de92-3a57-48e3-b899-f3e5cd515cca None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:54.861544 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260de92-3a57-48e3-b899-f3e5cd515cca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:54.862096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 22/41] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:54.863603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497060be-2406-40c6-951d-c608fbced0e9 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:54.863930 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497060be-2406-40c6-951d-c608fbced0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:54.864289 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497060be-2406-40c6-951d-c608fbced0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:54.864637 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260de92-3a57-48e3-b899-f3e5cd515cca None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:54.891699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-497060be-2406-40c6-951d-c608fbced0e9 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:54.893640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-497060be-2406-40c6-951d-c608fbced0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:54.962530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 20/42] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 14:49:54 2022] POST /identity/v3/roles => generated 223 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:49:54.964080 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260de92-3a57-48e3-b899-f3e5cd515cca None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:55.006634 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 23/43] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:49:55.046306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497060be-2406-40c6-951d-c608fbced0e9 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:55.145454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260de92-3a57-48e3-b899-f3e5cd515cca None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:55.188721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:55.195698 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50683ede-5252-485f-a22d-49a76a92b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:55.197331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50683ede-5252-485f-a22d-49a76a92b1a2 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:55.197786 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50683ede-5252-485f-a22d-49a76a92b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:55.198225 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50683ede-5252-485f-a22d-49a76a92b1a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:55.199387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50683ede-5252-485f-a22d-49a76a92b1a2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:55.205499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50683ede-5252-485f-a22d-49a76a92b1a2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:55.217265 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497060be-2406-40c6-951d-c608fbced0e9 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:55.281324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea431861-c2ed-4ec3-984a-4e308734c3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:55.282500 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea431861-c2ed-4ec3-984a-4e308734c3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:55.282915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea431861-c2ed-4ec3-984a-4e308734c3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:55.283319 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea431861-c2ed-4ec3-984a-4e308734c3d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:55.285250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea431861-c2ed-4ec3-984a-4e308734c3d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:55.287536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea431861-c2ed-4ec3-984a-4e308734c3d7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:55.290883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 24/44] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:49:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:55.299493 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea431861-c2ed-4ec3-984a-4e308734c3d7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:55.328088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 21/45] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:49:54 2022] POST /identity/v3/projects => generated 301 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:49:55.454269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea431861-c2ed-4ec3-984a-4e308734c3d7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:55.507180 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14ee6-ae2b-49b5-ba98-9b5c2f81301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:55.508488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14ee6-ae2b-49b5-ba98-9b5c2f81301a None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:55.509038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14ee6-ae2b-49b5-ba98-9b5c2f81301a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:55.509125 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14ee6-ae2b-49b5-ba98-9b5c2f81301a None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:55.512848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14ee6-ae2b-49b5-ba98-9b5c2f81301a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:55.512848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14ee6-ae2b-49b5-ba98-9b5c2f81301a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:55.531225 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:55.896969 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 25/46] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:49:55 2022] POST /identity/v3/projects => generated 312 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:49:56.763680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a88b08-caee-408a-af74-2f1aeabddfa1 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:56.764366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a88b08-caee-408a-af74-2f1aeabddfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:56.764366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a88b08-caee-408a-af74-2f1aeabddfa1 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:56.765665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 22/47] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:56.771336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957cfc2-e50d-40a3-ad58-b5424f917040 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:56.771787 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957cfc2-e50d-40a3-ad58-b5424f917040 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:56.771911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957cfc2-e50d-40a3-ad58-b5424f917040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:56.791519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b957cfc2-e50d-40a3-ad58-b5424f917040 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:56.793072 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b957cfc2-e50d-40a3-ad58-b5424f917040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:56.875689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 26/48] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:49:56.900022 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a88b08-caee-408a-af74-2f1aeabddfa1 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:56.931234 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:57.018104 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a88b08-caee-408a-af74-2f1aeabddfa1 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:57.062817 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b4159-b8a2-4ad9-9386-590009fe75eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:57.063624 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4159-b8a2-4ad9-9386-590009fe75eb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:57.063909 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4159-b8a2-4ad9-9386-590009fe75eb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:57.064276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4159-b8a2-4ad9-9386-590009fe75eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:57.076336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b4159-b8a2-4ad9-9386-590009fe75eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1d8b4159-b8a2-4ad9-9386-590009fe75eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:49:57.083997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:49:57.085298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:49:57.086158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 23/49] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:49:57.092442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957cfc2-e50d-40a3-ad58-b5424f917040 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:57.202470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957cfc2-e50d-40a3-ad58-b5424f917040 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:57.229363 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:57.237793 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dea492-40e7-4252-ab9f-f2d56a181067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:57.238776 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dea492-40e7-4252-ab9f-f2d56a181067 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:57.239064 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dea492-40e7-4252-ab9f-f2d56a181067 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:57.244294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dea492-40e7-4252-ab9f-f2d56a181067 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:57.244294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dea492-40e7-4252-ab9f-f2d56a181067 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:49:57.244294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dea492-40e7-4252-ab9f-f2d56a181067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:57.244294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dea492-40e7-4252-ab9f-f2d56a181067 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:57.253762 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 27/50] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:57.260626 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b4159-b8a2-4ad9-9386-590009fe75eb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:57.374449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:57.390451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b4159-b8a2-4ad9-9386-590009fe75eb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:57.432434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486f822-3c3c-4ad8-89fc-77f92f19895d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:57.434015 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486f822-3c3c-4ad8-89fc-77f92f19895d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:57.434015 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486f822-3c3c-4ad8-89fc-77f92f19895d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:57.434015 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486f822-3c3c-4ad8-89fc-77f92f19895d None admin] PATH_INFO: `/v3/users/b841e83eb2b74ca1b301a4644bbdbe93` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:57.435241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486f822-3c3c-4ad8-89fc-77f92f19895d None admin] RBAC: Authorizing `identity:get_user(user_id=b841e83eb2b74ca1b301a4644bbdbe93)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:57.436909 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486f822-3c3c-4ad8-89fc-77f92f19895d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:57.438417 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 24/51] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:49:57 2022] GET /identity/v3/users/b841e83eb2b74ca1b301a4644bbdbe93 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:49:57.443982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dea492-40e7-4252-ab9f-f2d56a181067 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:57.564200 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dea492-40e7-4252-ab9f-f2d56a181067 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:57.604011 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142385e-2c0b-4249-bc7f-db8bab7900dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:57.604856 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142385e-2c0b-4249-bc7f-db8bab7900dc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:57.605140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142385e-2c0b-4249-bc7f-db8bab7900dc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:57.605457 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142385e-2c0b-4249-bc7f-db8bab7900dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:57.606675 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142385e-2c0b-4249-bc7f-db8bab7900dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:57.608197 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142385e-2c0b-4249-bc7f-db8bab7900dc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:57.609860 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 28/52] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:49:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:57.615864 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486f822-3c3c-4ad8-89fc-77f92f19895d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:57.742522 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486f822-3c3c-4ad8-89fc-77f92f19895d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:57.783865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe86d2f-9eca-4097-8cc6-b54a46112239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:57.784598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe86d2f-9eca-4097-8cc6-b54a46112239 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:57.784783 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe86d2f-9eca-4097-8cc6-b54a46112239 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:57.785074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe86d2f-9eca-4097-8cc6-b54a46112239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:57.785936 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe86d2f-9eca-4097-8cc6-b54a46112239 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.domain.id=default, user.id=b841e83eb2b74ca1b301a4644bbdbe93) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:49:57.786260 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe86d2f-9eca-4097-8cc6-b54a46112239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:57.787748 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe86d2f-9eca-4097-8cc6-b54a46112239 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:57.794829 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 25/53] 10.222.0.88 () {56 vars in 1322 bytes} [Thu Jun 16 14:49:57 2022] GET /identity/v3/role_assignments?user.id=b841e83eb2b74ca1b301a4644bbdbe93&scope.domain.id=default&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 530 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:58.172669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5c2ef-ef18-4dcd-998f-cfc04f8c0e92 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:58.173219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5c2ef-ef18-4dcd-998f-cfc04f8c0e92 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:58.173219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5c2ef-ef18-4dcd-998f-cfc04f8c0e92 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:58.174664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 29/54] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:58.181789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9504-e839-421b-a854-2f0a470e16d2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:58.182126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9504-e839-421b-a854-2f0a470e16d2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:58.182483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9504-e839-421b-a854-2f0a470e16d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:58.200256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a24f9504-e839-421b-a854-2f0a470e16d2 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:58.201799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a24f9504-e839-421b-a854-2f0a470e16d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:58.216970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:58.284853 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 26/55] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:49:58.309339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5c2ef-ef18-4dcd-998f-cfc04f8c0e92 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:58.376198 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:58.428738 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5c2ef-ef18-4dcd-998f-cfc04f8c0e92 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:58.467494 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde8d13-7b62-4ec9-b5bb-aecba27acd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:58.468345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde8d13-7b62-4ec9-b5bb-aecba27acd88 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:58.468645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde8d13-7b62-4ec9-b5bb-aecba27acd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:58.468933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde8d13-7b62-4ec9-b5bb-aecba27acd88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:58.470268 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde8d13-7b62-4ec9-b5bb-aecba27acd88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:58.471753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde8d13-7b62-4ec9-b5bb-aecba27acd88 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:58.473499 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 30/56] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:49:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:49:58.479869 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f9504-e839-421b-a854-2f0a470e16d2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:58.612542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f9504-e839-421b-a854-2f0a470e16d2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:58.654210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdb50f-3353-4618-9d8d-abdf4f5a2dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:58.654939 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdb50f-3353-4618-9d8d-abdf4f5a2dda None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:58.655285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdb50f-3353-4618-9d8d-abdf4f5a2dda None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:58.655615 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdb50f-3353-4618-9d8d-abdf4f5a2dda None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:58.656425 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdb50f-3353-4618-9d8d-abdf4f5a2dda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:58.657997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdb50f-3353-4618-9d8d-abdf4f5a2dda None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:58.782027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:58.788360 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 27/57] 10.222.0.88 () {60 vars in 1140 bytes} [Thu Jun 16 14:49:58 2022] POST /identity/v3/projects => generated 298 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:50:01.100812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad35ebd-6c88-45ad-9fe2-23c21aa3ff9a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:01.101938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad35ebd-6c88-45ad-9fe2-23c21aa3ff9a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:01.101938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad35ebd-6c88-45ad-9fe2-23c21aa3ff9a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:01.103261 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 31/58] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:01.109026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc8bd2-651c-45e1-9878-338b298bfe50 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:01.109550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc8bd2-651c-45e1-9878-338b298bfe50 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:01.109646 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc8bd2-651c-45e1-9878-338b298bfe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:01.128791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7fbc8bd2-651c-45e1-9878-338b298bfe50 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:01.130028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7fbc8bd2-651c-45e1-9878-338b298bfe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:01.223176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 28/59] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:01.249464 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad35ebd-6c88-45ad-9fe2-23c21aa3ff9a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:01.371567 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad35ebd-6c88-45ad-9fe2-23c21aa3ff9a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:01.399539 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:01.414668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89fc5b-51cf-4efb-845a-52500243374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:01.415562 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89fc5b-51cf-4efb-845a-52500243374d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:01.415977 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89fc5b-51cf-4efb-845a-52500243374d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:01.416290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89fc5b-51cf-4efb-845a-52500243374d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:01.417588 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89fc5b-51cf-4efb-845a-52500243374d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:01.419078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89fc5b-51cf-4efb-845a-52500243374d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:01.421324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 32/60] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:01.427867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc8bd2-651c-45e1-9878-338b298bfe50 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:01.544832 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc8bd2-651c-45e1-9878-338b298bfe50 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:01.576070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:01.582174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba1aec-275f-45af-aad5-a8729358b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:01.582856 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba1aec-275f-45af-aad5-a8729358b002 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:01.583145 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba1aec-275f-45af-aad5-a8729358b002 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:01.583431 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba1aec-275f-45af-aad5-a8729358b002 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:01.584163 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba1aec-275f-45af-aad5-a8729358b002 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:01.586385 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba1aec-275f-45af-aad5-a8729358b002 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:01.702069 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 29/61] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:50:01 2022] POST /identity/v3/users => generated 275 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:50:05.041133 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27164ba6-8228-4281-b693-dc40d5fa01ad None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.042231 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27164ba6-8228-4281-b693-dc40d5fa01ad None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.042736 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27164ba6-8228-4281-b693-dc40d5fa01ad None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.044411 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 33/62] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:05.050174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f9da-4436-4615-86dc-4e0fc7e03c5a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.051111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f9da-4436-4615-86dc-4e0fc7e03c5a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.051597 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f9da-4436-4615-86dc-4e0fc7e03c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.078911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6aa8f9da-4436-4615-86dc-4e0fc7e03c5a None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:05.080121 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6aa8f9da-4436-4615-86dc-4e0fc7e03c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:05.211651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 30/63] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:05.254622 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27164ba6-8228-4281-b693-dc40d5fa01ad None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:05.478791 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27164ba6-8228-4281-b693-dc40d5fa01ad None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:05.569652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505b0af-e1a2-4d1c-916e-6f77df958a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:05.571279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b0af-e1a2-4d1c-916e-6f77df958a7c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.571887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b0af-e1a2-4d1c-916e-6f77df958a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.572808 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b0af-e1a2-4d1c-916e-6f77df958a7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.610241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b0af-e1a2-4d1c-916e-6f77df958a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d505b0af-e1a2-4d1c-916e-6f77df958a7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:05.628327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:05.629814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:05.630730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:05.634010 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 34/64] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:05.640491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f9da-4436-4615-86dc-4e0fc7e03c5a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:05.653104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703b327-776e-4e0f-8801-3cf979950e5e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.653395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703b327-776e-4e0f-8801-3cf979950e5e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.653866 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703b327-776e-4e0f-8801-3cf979950e5e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.655543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 35/65] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:05.661302 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355cae-ffb2-4f94-a894-6a34dec9cd67 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.661797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355cae-ffb2-4f94-a894-6a34dec9cd67 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.662223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355cae-ffb2-4f94-a894-6a34dec9cd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.696131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ed355cae-ffb2-4f94-a894-6a34dec9cd67 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:05.699610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ed355cae-ffb2-4f94-a894-6a34dec9cd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:05.780214 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f9da-4436-4615-86dc-4e0fc7e03c5a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:05.781308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:05.799883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:05.812382 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 36/66] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:50:05.814320 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540cd95-327f-44a9-9433-83a46a62feee None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.814641 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540cd95-327f-44a9-9433-83a46a62feee None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.815134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540cd95-327f-44a9-9433-83a46a62feee None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.822888 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 37/67] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:05.824549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d69437-1ba7-43d2-98d1-0a2e3439296b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.824996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d69437-1ba7-43d2-98d1-0a2e3439296b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.825314 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03100a32-b157-4e4b-bf40-a2beec0d8acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:05.825723 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d69437-1ba7-43d2-98d1-0a2e3439296b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.826083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100a32-b157-4e4b-bf40-a2beec0d8acc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.826429 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100a32-b157-4e4b-bf40-a2beec0d8acc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.826773 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03100a32-b157-4e4b-bf40-a2beec0d8acc None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.827604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03100a32-b157-4e4b-bf40-a2beec0d8acc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:05.827887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 38/68] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:05.828028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03100a32-b157-4e4b-bf40-a2beec0d8acc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:05.829303 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91d3ed-db32-4635-83b8-60d3cbdb21d6 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.829665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03100a32-b157-4e4b-bf40-a2beec0d8acc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:05.829900 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91d3ed-db32-4635-83b8-60d3cbdb21d6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.830853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91d3ed-db32-4635-83b8-60d3cbdb21d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.844393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 31/69] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:05.844393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd4fa9-b23a-4181-8e8a-0d1fcf2aaa29 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:05.844393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd4fa9-b23a-4181-8e8a-0d1fcf2aaa29 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:05.844393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd4fa9-b23a-4181-8e8a-0d1fcf2aaa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:05.861217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-aa91d3ed-db32-4635-83b8-60d3cbdb21d6 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:05.863089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-94cd4fa9-b23a-4181-8e8a-0d1fcf2aaa29 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:05.863707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-aa91d3ed-db32-4635-83b8-60d3cbdb21d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:05.864658 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-94cd4fa9-b23a-4181-8e8a-0d1fcf2aaa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:05.946239 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 39/70] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:05.946584 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91d3ed-db32-4635-83b8-60d3cbdb21d6 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:05.968381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 32/71] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:05.969622 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd4fa9-b23a-4181-8e8a-0d1fcf2aaa29 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:06.094463 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91d3ed-db32-4635-83b8-60d3cbdb21d6 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:06.119418 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:06.150946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0080a0-b37e-4234-b738-917bbac83077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:06.151402 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd4fa9-b23a-4181-8e8a-0d1fcf2aaa29 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:06.152096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0080a0-b37e-4234-b738-917bbac83077 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.152299 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0080a0-b37e-4234-b738-917bbac83077 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.152698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0080a0-b37e-4234-b738-917bbac83077 None admin] PATH_INFO: `/v3/users/b841e83eb2b74ca1b301a4644bbdbe93` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.155146 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0080a0-b37e-4234-b738-917bbac83077 None admin] RBAC: Authorizing `identity:get_user(user_id=b841e83eb2b74ca1b301a4644bbdbe93)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:06.157004 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0080a0-b37e-4234-b738-917bbac83077 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:06.159933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 40/72] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:05 2022] GET /identity/v3/users/b841e83eb2b74ca1b301a4644bbdbe93 => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:06.160554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0080a0-b37e-4234-b738-917bbac83077 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:06.166334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:06.212970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dbadc-747f-4e74-85ca-41375919b5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:06.214327 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dbadc-747f-4e74-85ca-41375919b5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.214838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dbadc-747f-4e74-85ca-41375919b5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.215370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dbadc-747f-4e74-85ca-41375919b5e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.234380 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7dbadc-747f-4e74-85ca-41375919b5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-0c7dbadc-747f-4e74-85ca-41375919b5e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:06.249879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:06.251110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:06.252171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 33/73] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:06.253574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dbadc-747f-4e74-85ca-41375919b5e9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:06.337603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0080a0-b37e-4234-b738-917bbac83077 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:06.396155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b7892-dc0d-4e57-acc9-4616f34460da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:06.397363 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b7892-dc0d-4e57-acc9-4616f34460da None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.397911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b7892-dc0d-4e57-acc9-4616f34460da None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.398838 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b7892-dc0d-4e57-acc9-4616f34460da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.413297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dbadc-747f-4e74-85ca-41375919b5e9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:06.415297 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b7892-dc0d-4e57-acc9-4616f34460da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-758b7892-dc0d-4e57-acc9-4616f34460da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:06.430349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:06.432432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:06.434026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 41/74] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:50:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:06.435297 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b7892-dc0d-4e57-acc9-4616f34460da None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:06.458763 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4f44a-4b57-4eb8-894f-4debef715e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:06.460142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f44a-4b57-4eb8-894f-4debef715e9a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.461350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f44a-4b57-4eb8-894f-4debef715e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.461859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f44a-4b57-4eb8-894f-4debef715e9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.480451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4f44a-4b57-4eb8-894f-4debef715e9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-48a4f44a-4b57-4eb8-894f-4debef715e9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:06.492589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:06.494627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:06.495581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 34/75] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:06.496296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4f44a-4b57-4eb8-894f-4debef715e9a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:06.501881 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:06.542027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:06.598727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b7892-dc0d-4e57-acc9-4616f34460da None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:06.643128 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd87f6-38a1-456d-b54f-365b5be67207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:06.644144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd87f6-38a1-456d-b54f-365b5be67207 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.644144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd87f6-38a1-456d-b54f-365b5be67207 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.645164 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd87f6-38a1-456d-b54f-365b5be67207 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.650632 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4f44a-4b57-4eb8-894f-4debef715e9a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:06.706906 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f3d93-2ccd-44e7-9bf4-4588135ce39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:06.707953 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f3d93-2ccd-44e7-9bf4-4588135ce39d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.708487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f3d93-2ccd-44e7-9bf4-4588135ce39d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.708957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f3d93-2ccd-44e7-9bf4-4588135ce39d None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.710240 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f3d93-2ccd-44e7-9bf4-4588135ce39d None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:06.710792 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f3d93-2ccd-44e7-9bf4-4588135ce39d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:06.713276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f3d93-2ccd-44e7-9bf4-4588135ce39d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:06.725040 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 35/76] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:06.726626 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dc151-53ce-4d19-94e4-07cf58f8bea4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.727144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dc151-53ce-4d19-94e4-07cf58f8bea4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.727274 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dc151-53ce-4d19-94e4-07cf58f8bea4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.732063 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 36/77] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:06.732063 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dc151-53ce-4d19-94e4-07cf58f8bea4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:06.749283 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd87f6-38a1-456d-b54f-365b5be67207 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:06.750880 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd87f6-38a1-456d-b54f-365b5be67207 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:06.800662 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:06.834917 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:06.846052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dc151-53ce-4d19-94e4-07cf58f8bea4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:06.876305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 42/78] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:06.877243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd87f6-38a1-456d-b54f-365b5be67207 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:06.887859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f9406-ae54-4a37-8677-8b6684bdf275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:06.888594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f9406-ae54-4a37-8677-8b6684bdf275 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.888886 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f9406-ae54-4a37-8677-8b6684bdf275 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.889172 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f9406-ae54-4a37-8677-8b6684bdf275 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.889941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f9406-ae54-4a37-8677-8b6684bdf275 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:06.890262 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f9406-ae54-4a37-8677-8b6684bdf275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:06.891759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f9406-ae54-4a37-8677-8b6684bdf275 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:06.901379 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 37/79] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:50:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:06.902126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f9406-ae54-4a37-8677-8b6684bdf275 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.021084 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd87f6-38a1-456d-b54f-365b5be67207 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.027381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f9406-ae54-4a37-8677-8b6684bdf275 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.062829 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96971cd-5c4b-4686-82ed-3a8f21053a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.063713 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96971cd-5c4b-4686-82ed-3a8f21053a84 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.064110 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96971cd-5c4b-4686-82ed-3a8f21053a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.064516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96971cd-5c4b-4686-82ed-3a8f21053a84 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.065392 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96971cd-5c4b-4686-82ed-3a8f21053a84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:07.065848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96971cd-5c4b-4686-82ed-3a8f21053a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:07.067332 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c71b67-b5b0-4a91-8c4d-22521b43c7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.067509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96971cd-5c4b-4686-82ed-3a8f21053a84 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:07.068202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c71b67-b5b0-4a91-8c4d-22521b43c7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.068525 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c71b67-b5b0-4a91-8c4d-22521b43c7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.068800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c71b67-b5b0-4a91-8c4d-22521b43c7d3 None admin] PATH_INFO: `/v3/users/d0a54582166343e5baada31da5b8f7d5` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.079246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 43/80] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:07.079246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9825-5cd1-49c0-bf56-aca4d4982a6d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.079246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9825-5cd1-49c0-bf56-aca4d4982a6d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.079708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e9825-5cd1-49c0-bf56-aca4d4982a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.087211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:07.096710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c71b67-b5b0-4a91-8c4d-22521b43c7d3 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a54582166343e5baada31da5b8f7d5)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:07.107649 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-418e9825-5cd1-49c0-bf56-aca4d4982a6d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:07.108141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c71b67-b5b0-4a91-8c4d-22521b43c7d3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:07.108304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-418e9825-5cd1-49c0-bf56-aca4d4982a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:07.108429 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 38/81] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:06 2022] GET /identity/v3/users/d0a54582166343e5baada31da5b8f7d5 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:07.108429 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c71b67-b5b0-4a91-8c4d-22521b43c7d3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.119873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:07.173022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 44/82] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:07.173635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e9825-5cd1-49c0-bf56-aca4d4982a6d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.221970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c71b67-b5b0-4a91-8c4d-22521b43c7d3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.261589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d00ee7-2733-4f43-bae6-b8f740007762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.262461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d00ee7-2733-4f43-bae6-b8f740007762 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.262754 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d00ee7-2733-4f43-bae6-b8f740007762 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.263041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d00ee7-2733-4f43-bae6-b8f740007762 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.264417 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d00ee7-2733-4f43-bae6-b8f740007762 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:07.266151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d00ee7-2733-4f43-bae6-b8f740007762 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:07.268385 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 39/83] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:07 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:07.269216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d00ee7-2733-4f43-bae6-b8f740007762 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.285968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e9825-5cd1-49c0-bf56-aca4d4982a6d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.322159 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:07.327670 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc54c7a-ba06-4830-8b47-537d14fbf4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.328926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc54c7a-ba06-4830-8b47-537d14fbf4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.332596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc54c7a-ba06-4830-8b47-537d14fbf4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.332596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc54c7a-ba06-4830-8b47-537d14fbf4c4 None admin] PATH_INFO: `/v3/users/d0a54582166343e5baada31da5b8f7d5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.332596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc54c7a-ba06-4830-8b47-537d14fbf4c4 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a54582166343e5baada31da5b8f7d5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:07.332596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc54c7a-ba06-4830-8b47-537d14fbf4c4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:07.334391 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 45/84] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:07 2022] GET /identity/v3/users/d0a54582166343e5baada31da5b8f7d5 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:07.336200 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc54c7a-ba06-4830-8b47-537d14fbf4c4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.377441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:07.402236 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d00ee7-2733-4f43-bae6-b8f740007762 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.442843 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe24746-eeb0-4242-9e33-aef3b6a79ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.443719 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe24746-eeb0-4242-9e33-aef3b6a79ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.444079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe24746-eeb0-4242-9e33-aef3b6a79ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.444445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe24746-eeb0-4242-9e33-aef3b6a79ca1 None admin] PATH_INFO: `/v3/users/d0a54582166343e5baada31da5b8f7d5` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.445761 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe24746-eeb0-4242-9e33-aef3b6a79ca1 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a54582166343e5baada31da5b8f7d5)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:07.447438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe24746-eeb0-4242-9e33-aef3b6a79ca1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:07.449261 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 40/85] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:07 2022] GET /identity/v3/users/d0a54582166343e5baada31da5b8f7d5 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:07.450558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe24746-eeb0-4242-9e33-aef3b6a79ca1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.494442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc54c7a-ba06-4830-8b47-537d14fbf4c4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.535883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11258c6-dd5d-4689-af91-45fb7a98474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.536568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11258c6-dd5d-4689-af91-45fb7a98474a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.536863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11258c6-dd5d-4689-af91-45fb7a98474a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.537145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11258c6-dd5d-4689-af91-45fb7a98474a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.582840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe24746-eeb0-4242-9e33-aef3b6a79ca1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.619444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:07.622298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11258c6-dd5d-4689-af91-45fb7a98474a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:07.624138 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11258c6-dd5d-4689-af91-45fb7a98474a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:07.628536 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd0bb2-dcbb-4723-a76b-ca26812e999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.629417 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd0bb2-dcbb-4723-a76b-ca26812e999e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.630094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd0bb2-dcbb-4723-a76b-ca26812e999e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.630576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd0bb2-dcbb-4723-a76b-ca26812e999e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.632978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd0bb2-dcbb-4723-a76b-ca26812e999e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:07.635407 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd0bb2-dcbb-4723-a76b-ca26812e999e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:07.638965 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 41/86] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:07.639176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd0bb2-dcbb-4723-a76b-ca26812e999e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.664868 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:07.753683 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 46/87] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:07.754434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11258c6-dd5d-4689-af91-45fb7a98474a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.774455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd0bb2-dcbb-4723-a76b-ca26812e999e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.822210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454bd62-fe7f-48a3-9020-beadeaa157ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.822872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454bd62-fe7f-48a3-9020-beadeaa157ac None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.823167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454bd62-fe7f-48a3-9020-beadeaa157ac None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.823522 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454bd62-fe7f-48a3-9020-beadeaa157ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.824328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454bd62-fe7f-48a3-9020-beadeaa157ac None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de, user.id=b841e83eb2b74ca1b301a4644bbdbe93) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:07.824648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454bd62-fe7f-48a3-9020-beadeaa157ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:07.826101 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454bd62-fe7f-48a3-9020-beadeaa157ac None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:07.835093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 42/88] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:07 2022] GET /identity/v3/role_assignments?user.id=b841e83eb2b74ca1b301a4644bbdbe93&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:07.837127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454bd62-fe7f-48a3-9020-beadeaa157ac None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.923222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:07.928979 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11258c6-dd5d-4689-af91-45fb7a98474a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.974957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:07.989913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247376bb-4c73-45dc-85de-dcdbf68b1d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.990962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247376bb-4c73-45dc-85de-dcdbf68b1d88 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.991388 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247376bb-4c73-45dc-85de-dcdbf68b1d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.992480 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247376bb-4c73-45dc-85de-dcdbf68b1d88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.005276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454bd62-fe7f-48a3-9020-beadeaa157ac None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:08.067000 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe18dc2-0d17-4d7b-ab60-8eec14af0d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:08.067000 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe18dc2-0d17-4d7b-ab60-8eec14af0d8d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:08.067000 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe18dc2-0d17-4d7b-ab60-8eec14af0d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:08.067000 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe18dc2-0d17-4d7b-ab60-8eec14af0d8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.080265 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247376bb-4c73-45dc-85de-dcdbf68b1d88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:08.082382 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247376bb-4c73-45dc-85de-dcdbf68b1d88 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:08.159430 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe18dc2-0d17-4d7b-ab60-8eec14af0d8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:08.161312 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe18dc2-0d17-4d7b-ab60-8eec14af0d8d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:08.202661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:08.214933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 47/89] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:08.214933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247376bb-4c73-45dc-85de-dcdbf68b1d88 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:08.268180 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:08.302128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 43/90] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:08.303870 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe18dc2-0d17-4d7b-ab60-8eec14af0d8d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:08.364828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247376bb-4c73-45dc-85de-dcdbf68b1d88 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:08.415432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe66a12-ec59-4dbd-bbd4-ee87fcbb98af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:08.416708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe66a12-ec59-4dbd-bbd4-ee87fcbb98af None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:08.417188 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe66a12-ec59-4dbd-bbd4-ee87fcbb98af None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:08.420143 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe66a12-ec59-4dbd-bbd4-ee87fcbb98af None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.420143 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe66a12-ec59-4dbd-bbd4-ee87fcbb98af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:08.421402 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe66a12-ec59-4dbd-bbd4-ee87fcbb98af None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:08.434037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe18dc2-0d17-4d7b-ab60-8eec14af0d8d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:08.471163 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d420bb3-5c07-4ee4-bdf7-a43a549bd042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:08.472035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d420bb3-5c07-4ee4-bdf7-a43a549bd042 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:08.485883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d420bb3-5c07-4ee4-bdf7-a43a549bd042 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:08.485883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d420bb3-5c07-4ee4-bdf7-a43a549bd042 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.485883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d420bb3-5c07-4ee4-bdf7-a43a549bd042 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:08.485883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d420bb3-5c07-4ee4-bdf7-a43a549bd042 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:08.485883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 44/91] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:08 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:08.485883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d420bb3-5c07-4ee4-bdf7-a43a549bd042 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:08.543000 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:08.601379 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d420bb3-5c07-4ee4-bdf7-a43a549bd042 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:08.616276 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 48/92] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:50:08 2022] POST /identity/v3/users => generated 289 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:50:08.616276 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe66a12-ec59-4dbd-bbd4-ee87fcbb98af None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:08.671229 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6748189-f890-4c9a-a9d4-9737d6b13025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:08.672887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6748189-f890-4c9a-a9d4-9737d6b13025 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:08.673733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6748189-f890-4c9a-a9d4-9737d6b13025 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:08.674282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6748189-f890-4c9a-a9d4-9737d6b13025 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.676931 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6748189-f890-4c9a-a9d4-9737d6b13025 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:08.680731 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6748189-f890-4c9a-a9d4-9737d6b13025 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:08.684533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 45/93] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:08 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:08.686204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6748189-f890-4c9a-a9d4-9737d6b13025 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:08.720407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:08.805271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe66a12-ec59-4dbd-bbd4-ee87fcbb98af None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:08.852874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6748189-f890-4c9a-a9d4-9737d6b13025 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:08.857992 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94935032-840e-4910-8c41-963ef1453573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:08.858876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94935032-840e-4910-8c41-963ef1453573 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:08.859163 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94935032-840e-4910-8c41-963ef1453573 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:08.859446 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94935032-840e-4910-8c41-963ef1453573 None admin] PATH_INFO: `/v3/projects/5c254b7a6c224c6abc8b7a4b9c28db47` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.865723 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94935032-840e-4910-8c41-963ef1453573 None admin] RBAC: Authorizing `identity:get_project(project_id=5c254b7a6c224c6abc8b7a4b9c28db47)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:08.868993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94935032-840e-4910-8c41-963ef1453573 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:08.871708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 49/94] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:08 2022] GET /identity/v3/projects/5c254b7a6c224c6abc8b7a4b9c28db47 => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:08.872543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94935032-840e-4910-8c41-963ef1453573 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:08.879354 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:08.925884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ed34a-87ef-41ac-83c9-cc7c456181ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:08.927482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ed34a-87ef-41ac-83c9-cc7c456181ae None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:08.928310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ed34a-87ef-41ac-83c9-cc7c456181ae None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:08.929562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ed34a-87ef-41ac-83c9-cc7c456181ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.931706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ed34a-87ef-41ac-83c9-cc7c456181ae None admin] RBAC: Adding query filter params (role.id=4498e695105347909981a684f613b6a2, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de, user.id=d0a54582166343e5baada31da5b8f7d5) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:08.932669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ed34a-87ef-41ac-83c9-cc7c456181ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:08.935541 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ed34a-87ef-41ac-83c9-cc7c456181ae None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:08.948321 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 46/95] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:08 2022] GET /identity/v3/role_assignments?user.id=d0a54582166343e5baada31da5b8f7d5&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=4498e695105347909981a684f613b6a2 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:08.948688 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ed34a-87ef-41ac-83c9-cc7c456181ae None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:09.083704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94935032-840e-4910-8c41-963ef1453573 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:09.120381 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:09.149666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc62df7-2acf-4261-8db0-b25462622362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:09.150827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc62df7-2acf-4261-8db0-b25462622362 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:09.151424 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc62df7-2acf-4261-8db0-b25462622362 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:09.151927 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc62df7-2acf-4261-8db0-b25462622362 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:09.153867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc62df7-2acf-4261-8db0-b25462622362 None admin] RBAC: Adding query filter params (role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de, user.id=d0a54582166343e5baada31da5b8f7d5) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:09.154256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc62df7-2acf-4261-8db0-b25462622362 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:09.158601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc62df7-2acf-4261-8db0-b25462622362 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:09.162179 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ed34a-87ef-41ac-83c9-cc7c456181ae None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:09.176388 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 50/96] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:08 2022] GET /identity/v3/role_assignments?user.id=d0a54582166343e5baada31da5b8f7d5&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:09.222760 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4488a0-facb-4725-9dca-89c1334cca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:09.223857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4488a0-facb-4725-9dca-89c1334cca2f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:09.224535 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4488a0-facb-4725-9dca-89c1334cca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:09.225174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4488a0-facb-4725-9dca-89c1334cca2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:09.227853 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4488a0-facb-4725-9dca-89c1334cca2f None admin] RBAC: Adding query filter params (role.id=4498e695105347909981a684f613b6a2, scope.project.id=5c254b7a6c224c6abc8b7a4b9c28db47, user.id=d0a54582166343e5baada31da5b8f7d5) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:09.228746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4488a0-facb-4725-9dca-89c1334cca2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:09.231727 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4488a0-facb-4725-9dca-89c1334cca2f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:09.246855 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 47/97] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:08 2022] GET /identity/v3/role_assignments?user.id=d0a54582166343e5baada31da5b8f7d5&scope.project.id=5c254b7a6c224c6abc8b7a4b9c28db47&role.id=4498e695105347909981a684f613b6a2 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:11.402269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8130e-0a3c-4c6a-ad18-0667d43f8077 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:11.403328 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8130e-0a3c-4c6a-ad18-0667d43f8077 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:11.403328 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8130e-0a3c-4c6a-ad18-0667d43f8077 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:11.404474 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 51/98] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:11.410389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f459b5e-8df2-481a-bce2-b3a82ca466c0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:11.411215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f459b5e-8df2-481a-bce2-b3a82ca466c0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:11.411215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f459b5e-8df2-481a-bce2-b3a82ca466c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:11.434457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6f459b5e-8df2-481a-bce2-b3a82ca466c0 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:11.435664 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6f459b5e-8df2-481a-bce2-b3a82ca466c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:11.509515 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:11.590882 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 48/99] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:50:11.656642 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8130e-0a3c-4c6a-ad18-0667d43f8077 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:11.874740 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8130e-0a3c-4c6a-ad18-0667d43f8077 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:11.945254 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a10c0-d7a1-472f-a093-3048efe33a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:11.946535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a10c0-d7a1-472f-a093-3048efe33a3e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:11.947043 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a10c0-d7a1-472f-a093-3048efe33a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:11.947552 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a10c0-d7a1-472f-a093-3048efe33a3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:11.966266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a10c0-d7a1-472f-a093-3048efe33a3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-043a10c0-d7a1-472f-a093-3048efe33a3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:11.981681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:11.983009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:11.984298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 52/100] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:11.994455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f459b5e-8df2-481a-bce2-b3a82ca466c0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:12.297591 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f459b5e-8df2-481a-bce2-b3a82ca466c0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:12.370135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badb8a7-a8b5-4aa0-87d4-16036b470537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:12.371192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badb8a7-a8b5-4aa0-87d4-16036b470537 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:12.371727 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badb8a7-a8b5-4aa0-87d4-16036b470537 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:12.372218 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badb8a7-a8b5-4aa0-87d4-16036b470537 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:12.373299 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badb8a7-a8b5-4aa0-87d4-16036b470537 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:12.373832 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badb8a7-a8b5-4aa0-87d4-16036b470537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:12.375549 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badb8a7-a8b5-4aa0-87d4-16036b470537 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:12.389551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 49/101] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:12.399130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a10c0-d7a1-472f-a093-3048efe33a3e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:12.536021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf206216-16d1-4337-b8c5-831cf47fdcbb None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:12.537522 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf206216-16d1-4337-b8c5-831cf47fdcbb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:12.538966 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf206216-16d1-4337-b8c5-831cf47fdcbb None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:12.541910 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 50/102] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:12.548422 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:12.550898 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac97b5-6609-4b0c-982d-4c434bbb7138 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:12.551502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac97b5-6609-4b0c-982d-4c434bbb7138 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:12.552095 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac97b5-6609-4b0c-982d-4c434bbb7138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:12.596618 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d2ac97b5-6609-4b0c-982d-4c434bbb7138 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:12.598790 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d2ac97b5-6609-4b0c-982d-4c434bbb7138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:12.692010 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:12.731948 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a10c0-d7a1-472f-a093-3048efe33a3e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:12.829465 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 51/103] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:12.841661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550125-1956-4c98-89f5-0e7c2f3b15bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:12.843241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550125-1956-4c98-89f5-0e7c2f3b15bc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:12.843756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550125-1956-4c98-89f5-0e7c2f3b15bc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:12.844294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550125-1956-4c98-89f5-0e7c2f3b15bc None admin] PATH_INFO: `/v3/users/b841e83eb2b74ca1b301a4644bbdbe93` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:12.846489 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67550125-1956-4c98-89f5-0e7c2f3b15bc None admin] RBAC: Authorizing `identity:get_user(user_id=b841e83eb2b74ca1b301a4644bbdbe93)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:12.849298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67550125-1956-4c98-89f5-0e7c2f3b15bc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:12.853119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 53/104] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:12 2022] GET /identity/v3/users/b841e83eb2b74ca1b301a4644bbdbe93 => generated 247 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:12.863683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac97b5-6609-4b0c-982d-4c434bbb7138 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:12.883540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550125-1956-4c98-89f5-0e7c2f3b15bc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:13.101080 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac97b5-6609-4b0c-982d-4c434bbb7138 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:13.205151 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550125-1956-4c98-89f5-0e7c2f3b15bc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:13.234085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b435d-23b2-4875-ae86-d1fa3c231bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:13.235546 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b435d-23b2-4875-ae86-d1fa3c231bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:13.236057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b435d-23b2-4875-ae86-d1fa3c231bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:13.236779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b435d-23b2-4875-ae86-d1fa3c231bd3 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:13.239359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b435d-23b2-4875-ae86-d1fa3c231bd3 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:13.242572 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b435d-23b2-4875-ae86-d1fa3c231bd3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:13.245572 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 52/105] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:12 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:13.252669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b435d-23b2-4875-ae86-d1fa3c231bd3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:13.257650 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:13.302370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:13.316917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729cebf-c08a-4819-b8cc-4ab58b6bc656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:13.321147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729cebf-c08a-4819-b8cc-4ab58b6bc656 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:13.321147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729cebf-c08a-4819-b8cc-4ab58b6bc656 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:13.321147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729cebf-c08a-4819-b8cc-4ab58b6bc656 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:13.338989 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729cebf-c08a-4819-b8cc-4ab58b6bc656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1729cebf-c08a-4819-b8cc-4ab58b6bc656 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:13.348166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:13.349181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:13.350307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:13.351688 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 54/106] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 470 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:50:13.364122 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729cebf-c08a-4819-b8cc-4ab58b6bc656 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:13.469908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b435d-23b2-4875-ae86-d1fa3c231bd3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:13.526619 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729cebf-c08a-4819-b8cc-4ab58b6bc656 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:13.564767 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d932c5-3b77-414f-94f1-bf4ac3dcb3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:13.566300 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d932c5-3b77-414f-94f1-bf4ac3dcb3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:13.566798 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d932c5-3b77-414f-94f1-bf4ac3dcb3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:13.567350 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d932c5-3b77-414f-94f1-bf4ac3dcb3d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:13.568400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d932c5-3b77-414f-94f1-bf4ac3dcb3d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:13.568935 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d932c5-3b77-414f-94f1-bf4ac3dcb3d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:13.570764 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d932c5-3b77-414f-94f1-bf4ac3dcb3d0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:13.574237 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632b29d-4ed7-40b2-8559-e40f1ecce9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:13.575766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632b29d-4ed7-40b2-8559-e40f1ecce9ed None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:13.576307 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632b29d-4ed7-40b2-8559-e40f1ecce9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:13.576816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632b29d-4ed7-40b2-8559-e40f1ecce9ed None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/users/b841e83eb2b74ca1b301a4644bbdbe93/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:13.582121 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 55/107] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:13.582531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632b29d-4ed7-40b2-8559-e40f1ecce9ed None admin] RBAC: Authorizing `identity:create_grant(project_id=7c4956db4fa4484cbedbb6ce367d55de, user_id=b841e83eb2b74ca1b301a4644bbdbe93, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:13.584146 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efb50b-5d6e-4cd6-a01f-aab9dfa45f1b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:13.584775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efb50b-5d6e-4cd6-a01f-aab9dfa45f1b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:13.585772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efb50b-5d6e-4cd6-a01f-aab9dfa45f1b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:13.586295 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632b29d-4ed7-40b2-8559-e40f1ecce9ed None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:13.592582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 56/108] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:13.594656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efb50b-5d6e-4cd6-a01f-aab9dfa45f1b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:13.650805 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:13.708691 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 53/109] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:50:13 2022] PUT /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/users/b841e83eb2b74ca1b301a4644bbdbe93/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:50:13.711772 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9da113-f596-48f5-a107-5389504baf4e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:13.712310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9da113-f596-48f5-a107-5389504baf4e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:13.712822 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9da113-f596-48f5-a107-5389504baf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:13.715625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efb50b-5d6e-4cd6-a01f-aab9dfa45f1b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:13.774414 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8b9da113-f596-48f5-a107-5389504baf4e None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:13.776568 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8b9da113-f596-48f5-a107-5389504baf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:13.778933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d602e-d50d-4b72-afd6-b3888ff1e0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:13.780974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d602e-d50d-4b72-afd6-b3888ff1e0be None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:13.781707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d602e-d50d-4b72-afd6-b3888ff1e0be None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:13.782409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d602e-d50d-4b72-afd6-b3888ff1e0be None admin] PATH_INFO: `/v3/users/369fe16545434c1fa7f21a3e742ac0b9` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:13.811772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d602e-d50d-4b72-afd6-b3888ff1e0be None admin] RBAC: Authorizing `identity:get_user(user_id=369fe16545434c1fa7f21a3e742ac0b9)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:13.814788 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d602e-d50d-4b72-afd6-b3888ff1e0be None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:13.818505 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 57/110] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:13 2022] GET /identity/v3/users/369fe16545434c1fa7f21a3e742ac0b9 => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:13.827421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d602e-d50d-4b72-afd6-b3888ff1e0be None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:14.016937 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:14.029442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d602e-d50d-4b72-afd6-b3888ff1e0be None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:14.056079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 54/111] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:14.058481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404863a-8d7c-47b5-8389-1300e144535c None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.058990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404863a-8d7c-47b5-8389-1300e144535c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.059475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404863a-8d7c-47b5-8389-1300e144535c None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.063345 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 55/112] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:14.065442 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3020f4-0217-4c21-a884-05995cff3cd6 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.065941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3020f4-0217-4c21-a884-05995cff3cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.066443 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3020f4-0217-4c21-a884-05995cff3cd6 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.071038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 56/113] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:14.072382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d89bdc-b2b5-4b66-9d1a-28673e2e8703 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.072886 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d89bdc-b2b5-4b66-9d1a-28673e2e8703 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.073453 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d89bdc-b2b5-4b66-9d1a-28673e2e8703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.106383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-41d89bdc-b2b5-4b66-9d1a-28673e2e8703 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:14.107965 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0f25d-910b-49f6-abbd-3a0e456e10ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:14.108694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0f25d-910b-49f6-abbd-3a0e456e10ac None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.108916 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-41d89bdc-b2b5-4b66-9d1a-28673e2e8703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:14.109065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0f25d-910b-49f6-abbd-3a0e456e10ac None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.109258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0f25d-910b-49f6-abbd-3a0e456e10ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.140985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:14.234545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 57/114] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:14.237217 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62e482-b322-48d6-83c0-b05885819aba None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.238047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62e482-b322-48d6-83c0-b05885819aba None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.238759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62e482-b322-48d6-83c0-b05885819aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.259827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0f25d-910b-49f6-abbd-3a0e456e10ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:14.264146 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0f25d-910b-49f6-abbd-3a0e456e10ac None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:14.277185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ab62e482-b322-48d6-83c0-b05885819aba None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:14.282663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ab62e482-b322-48d6-83c0-b05885819aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:14.401835 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 58/115] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:14.409328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62e482-b322-48d6-83c0-b05885819aba None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:14.446998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:14.455860 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 58/116] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:50:14.456908 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0f25d-910b-49f6-abbd-3a0e456e10ac None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:14.544818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:14.570798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62e482-b322-48d6-83c0-b05885819aba None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:14.618507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0a6e4-a053-4bd4-bf26-1a8ac0b90f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:14.619439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0a6e4-a053-4bd4-bf26-1a8ac0b90f57 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.619756 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0a6e4-a053-4bd4-bf26-1a8ac0b90f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.620043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0a6e4-a053-4bd4-bf26-1a8ac0b90f57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.628690 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0f25d-910b-49f6-abbd-3a0e456e10ac None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:14.632293 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0a6e4-a053-4bd4-bf26-1a8ac0b90f57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-27c0a6e4-a053-4bd4-bf26-1a8ac0b90f57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:14.640743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:14.641641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:14.642547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 59/117] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:50:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:14.643220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b80c8e-03f1-4354-973f-d10588181f4e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.643924 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b80c8e-03f1-4354-973f-d10588181f4e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.643924 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b80c8e-03f1-4354-973f-d10588181f4e None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.645162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 60/118] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:14.646078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b80c8e-03f1-4354-973f-d10588181f4e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:14.686078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f892c-6365-40bc-ac1c-87a90daf49f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:14.687123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f892c-6365-40bc-ac1c-87a90daf49f7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.687622 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f892c-6365-40bc-ac1c-87a90daf49f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.688009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f892c-6365-40bc-ac1c-87a90daf49f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.689389 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f892c-6365-40bc-ac1c-87a90daf49f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:14.690860 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f892c-6365-40bc-ac1c-87a90daf49f7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:14.692854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 59/119] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:14.698733 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f892c-6365-40bc-ac1c-87a90daf49f7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:14.783281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:14.806417 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b80c8e-03f1-4354-973f-d10588181f4e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:14.843823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f892c-6365-40bc-ac1c-87a90daf49f7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:14.855882 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3fdba-cfd7-418d-8215-966687e1f790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:14.856735 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3fdba-cfd7-418d-8215-966687e1f790 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.857026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3fdba-cfd7-418d-8215-966687e1f790 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.857306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3fdba-cfd7-418d-8215-966687e1f790 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.874543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3fdba-cfd7-418d-8215-966687e1f790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-cad3fdba-cfd7-418d-8215-966687e1f790 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:14.885455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:14.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:14.887301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 61/120] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:14.887956 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3fdba-cfd7-418d-8215-966687e1f790 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:14.892014 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:14.913614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052b73d-f8d2-426e-b830-a40203134f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:14.915080 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b73d-f8d2-426e-b830-a40203134f0c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.915510 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b73d-f8d2-426e-b830-a40203134f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.916128 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b73d-f8d2-426e-b830-a40203134f0c None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.918534 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052b73d-f8d2-426e-b830-a40203134f0c None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:14.921218 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052b73d-f8d2-426e-b830-a40203134f0c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:14.924395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 60/121] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:14 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:14.926664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801016e9-aff2-4f37-b645-aee4b94a0839 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.927178 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801016e9-aff2-4f37-b645-aee4b94a0839 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.927685 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801016e9-aff2-4f37-b645-aee4b94a0839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.957876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-801016e9-aff2-4f37-b645-aee4b94a0839 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:14.959903 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-801016e9-aff2-4f37-b645-aee4b94a0839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:15.065399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3fdba-cfd7-418d-8215-966687e1f790 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:15.065640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 61/122] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:15.069084 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801016e9-aff2-4f37-b645-aee4b94a0839 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:15.123188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0f962-710d-4106-ab36-7a79dd94044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:15.123656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0f962-710d-4106-ab36-7a79dd94044b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:15.123930 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0f962-710d-4106-ab36-7a79dd94044b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:15.124855 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0f962-710d-4106-ab36-7a79dd94044b None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:15.125665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0f962-710d-4106-ab36-7a79dd94044b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:15.125980 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0f962-710d-4106-ab36-7a79dd94044b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:15.128614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0f962-710d-4106-ab36-7a79dd94044b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:15.153067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 62/123] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:50:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:15.153067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0f962-710d-4106-ab36-7a79dd94044b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:15.208087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:15.274061 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:15.286511 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801016e9-aff2-4f37-b645-aee4b94a0839 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:15.329035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0f962-710d-4106-ab36-7a79dd94044b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:15.336805 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05668064-5eaf-4d9e-9530-d76058940eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:15.337601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05668064-5eaf-4d9e-9530-d76058940eb6 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:15.337898 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05668064-5eaf-4d9e-9530-d76058940eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:15.338186 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05668064-5eaf-4d9e-9530-d76058940eb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:15.338947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05668064-5eaf-4d9e-9530-d76058940eb6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:15.340833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05668064-5eaf-4d9e-9530-d76058940eb6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:15.392989 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40232bb9-12a9-45d1-9168-e191360af25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:15.393760 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232bb9-12a9-45d1-9168-e191360af25c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:15.394050 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232bb9-12a9-45d1-9168-e191360af25c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:15.394330 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40232bb9-12a9-45d1-9168-e191360af25c None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:15.395360 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40232bb9-12a9-45d1-9168-e191360af25c None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:15.395679 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40232bb9-12a9-45d1-9168-e191360af25c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:15.397124 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40232bb9-12a9-45d1-9168-e191360af25c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:15.411585 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 63/124] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:15.412257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40232bb9-12a9-45d1-9168-e191360af25c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:15.468334 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 62/125] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:50:15 2022] POST /identity/v3/projects => generated 302 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:50:15.468334 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05668064-5eaf-4d9e-9530-d76058940eb6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:15.628209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40232bb9-12a9-45d1-9168-e191360af25c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:15.690179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05668064-5eaf-4d9e-9530-d76058940eb6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:15.693440 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:15.710454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe094f8b-552c-4d0b-8f32-6db65e5792c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:15.711764 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe094f8b-552c-4d0b-8f32-6db65e5792c2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:15.712267 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe094f8b-552c-4d0b-8f32-6db65e5792c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:15.713299 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:15.714203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe094f8b-552c-4d0b-8f32-6db65e5792c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:15.715673 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe094f8b-552c-4d0b-8f32-6db65e5792c2 None admin] RBAC: Adding query filter params (role.id=8e0ef0ea0d26497db73b62d43d4fda22, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de, user.id=369fe16545434c1fa7f21a3e742ac0b9) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:15.716157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe094f8b-552c-4d0b-8f32-6db65e5792c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:15.718419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe094f8b-552c-4d0b-8f32-6db65e5792c2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:15.727182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 64/126] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:15 2022] GET /identity/v3/role_assignments?user.id=369fe16545434c1fa7f21a3e742ac0b9&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=8e0ef0ea0d26497db73b62d43d4fda22 => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:15.728297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe094f8b-552c-4d0b-8f32-6db65e5792c2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:15.745449 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f434b99-0e50-41f5-9c6d-0ee4d95952ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:15.746784 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f434b99-0e50-41f5-9c6d-0ee4d95952ea None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:15.747279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f434b99-0e50-41f5-9c6d-0ee4d95952ea None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:15.747772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f434b99-0e50-41f5-9c6d-0ee4d95952ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:15.767500 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f434b99-0e50-41f5-9c6d-0ee4d95952ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-0f434b99-0e50-41f5-9c6d-0ee4d95952ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:15.778894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:15.779852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:15.780689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:15.783647 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 63/127] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:15.789182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f434b99-0e50-41f5-9c6d-0ee4d95952ea None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:15.923465 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe094f8b-552c-4d0b-8f32-6db65e5792c2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:15.977622 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f434b99-0e50-41f5-9c6d-0ee4d95952ea None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:16.011244 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30316bde-813d-4208-a1d0-d663f75a142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:16.013220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30316bde-813d-4208-a1d0-d663f75a142f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:16.014003 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30316bde-813d-4208-a1d0-d663f75a142f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:16.014439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30316bde-813d-4208-a1d0-d663f75a142f None admin] PATH_INFO: `/v3/users/d0a54582166343e5baada31da5b8f7d5` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:16.017894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30316bde-813d-4208-a1d0-d663f75a142f None admin] RBAC: Authorizing `identity:get_user(user_id=d0a54582166343e5baada31da5b8f7d5)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:16.020993 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30316bde-813d-4208-a1d0-d663f75a142f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:16.023074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 65/128] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:15 2022] GET /identity/v3/users/d0a54582166343e5baada31da5b8f7d5 => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:16.023943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30316bde-813d-4208-a1d0-d663f75a142f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:16.036469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8a049-4feb-4a76-8ec4-4a88a4a14bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:16.038749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8a049-4feb-4a76-8ec4-4a88a4a14bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:16.040388 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8a049-4feb-4a76-8ec4-4a88a4a14bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:16.041111 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8a049-4feb-4a76-8ec4-4a88a4a14bd5 None admin] PATH_INFO: `/v3/users/d0a54582166343e5baada31da5b8f7d5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:16.044938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8a049-4feb-4a76-8ec4-4a88a4a14bd5 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a54582166343e5baada31da5b8f7d5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:16.048482 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8a049-4feb-4a76-8ec4-4a88a4a14bd5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:16.056366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 64/129] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:15 2022] GET /identity/v3/users/d0a54582166343e5baada31da5b8f7d5 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:16.058050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8a049-4feb-4a76-8ec4-4a88a4a14bd5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:16.122337 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:16.155402 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:16.172199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30316bde-813d-4208-a1d0-d663f75a142f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:16.232790 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181cd6c-3c02-42ca-b6b3-1159fb19f5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:16.234052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cd6c-3c02-42ca-b6b3-1159fb19f5db None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:16.234690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cd6c-3c02-42ca-b6b3-1159fb19f5db None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:16.235320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181cd6c-3c02-42ca-b6b3-1159fb19f5db None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:16.236879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181cd6c-3c02-42ca-b6b3-1159fb19f5db None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:16.237576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181cd6c-3c02-42ca-b6b3-1159fb19f5db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:16.239922 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181cd6c-3c02-42ca-b6b3-1159fb19f5db None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:16.254239 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 66/130] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:16.255619 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181cd6c-3c02-42ca-b6b3-1159fb19f5db None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:16.267195 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8a049-4feb-4a76-8ec4-4a88a4a14bd5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:16.333495 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59850926-428d-4dfe-9639-fb8081d9aaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:16.335870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59850926-428d-4dfe-9639-fb8081d9aaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:16.336578 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59850926-428d-4dfe-9639-fb8081d9aaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:16.337369 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59850926-428d-4dfe-9639-fb8081d9aaf4 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:16.339791 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59850926-428d-4dfe-9639-fb8081d9aaf4 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:16.342863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59850926-428d-4dfe-9639-fb8081d9aaf4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:16.346874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 65/131] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:16 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:16.348668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59850926-428d-4dfe-9639-fb8081d9aaf4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:16.391494 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181cd6c-3c02-42ca-b6b3-1159fb19f5db None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:16.420559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:16.436721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e03c41-f841-42e7-bea3-03e58752a918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:16.437759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e03c41-f841-42e7-bea3-03e58752a918 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:16.438055 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e03c41-f841-42e7-bea3-03e58752a918 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:16.438348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e03c41-f841-42e7-bea3-03e58752a918 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:16.440051 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e03c41-f841-42e7-bea3-03e58752a918 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:16.441296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e03c41-f841-42e7-bea3-03e58752a918 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:16.448256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 67/132] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:16 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:16.448256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e03c41-f841-42e7-bea3-03e58752a918 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:16.619199 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59850926-428d-4dfe-9639-fb8081d9aaf4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:16.650232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:16.690341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e03c41-f841-42e7-bea3-03e58752a918 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:16.742523 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dec52a-c031-47e6-9467-561d5c0ad123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:16.743723 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dec52a-c031-47e6-9467-561d5c0ad123 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:16.744173 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dec52a-c031-47e6-9467-561d5c0ad123 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:16.744658 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dec52a-c031-47e6-9467-561d5c0ad123 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/users/d0a54582166343e5baada31da5b8f7d5/roles/f4c6e6a3b05a42abb56447797f7ade15` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:16.747409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a0118-89c0-4566-8a3d-384cc7741072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:16.748347 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a0118-89c0-4566-8a3d-384cc7741072 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:16.748644 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a0118-89c0-4566-8a3d-384cc7741072 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:16.749094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a0118-89c0-4566-8a3d-384cc7741072 None admin] PATH_INFO: `/v3/users/d0a54582166343e5baada31da5b8f7d5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:16.752155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a0118-89c0-4566-8a3d-384cc7741072 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a54582166343e5baada31da5b8f7d5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:16.754273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dec52a-c031-47e6-9467-561d5c0ad123 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c4956db4fa4484cbedbb6ce367d55de, user_id=d0a54582166343e5baada31da5b8f7d5, role_id=f4c6e6a3b05a42abb56447797f7ade15)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:16.755860 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a0118-89c0-4566-8a3d-384cc7741072 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:16.758523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 66/133] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:16 2022] GET /identity/v3/users/d0a54582166343e5baada31da5b8f7d5 => generated 275 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:16.759621 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a0118-89c0-4566-8a3d-384cc7741072 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:16.762584 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dec52a-c031-47e6-9467-561d5c0ad123 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:16.896304 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 68/134] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:50:16 2022] PUT /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/users/d0a54582166343e5baada31da5b8f7d5/roles/f4c6e6a3b05a42abb56447797f7ade15 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:50:16.899297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dec52a-c031-47e6-9467-561d5c0ad123 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:16.981003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a0118-89c0-4566-8a3d-384cc7741072 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:17.007882 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:17.120870 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dec52a-c031-47e6-9467-561d5c0ad123 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:17.133495 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612708f-7d07-4455-80e4-5ba9ec0f4764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:17.135251 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612708f-7d07-4455-80e4-5ba9ec0f4764 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:17.135749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612708f-7d07-4455-80e4-5ba9ec0f4764 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:17.136255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612708f-7d07-4455-80e4-5ba9ec0f4764 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/users/d0a54582166343e5baada31da5b8f7d5/roles/4498e695105347909981a684f613b6a2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:17.146905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612708f-7d07-4455-80e4-5ba9ec0f4764 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c4956db4fa4484cbedbb6ce367d55de, user_id=d0a54582166343e5baada31da5b8f7d5, role_id=4498e695105347909981a684f613b6a2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:17.149584 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612708f-7d07-4455-80e4-5ba9ec0f4764 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:17.198752 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7e476-ac7a-4032-bf43-420a93dbe475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:17.201249 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e476-ac7a-4032-bf43-420a93dbe475 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:17.202473 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e476-ac7a-4032-bf43-420a93dbe475 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:17.203620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e476-ac7a-4032-bf43-420a93dbe475 None admin] PATH_INFO: `/v3/projects/5c254b7a6c224c6abc8b7a4b9c28db47` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:17.206812 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7e476-ac7a-4032-bf43-420a93dbe475 None admin] RBAC: Authorizing `identity:get_project(project_id=5c254b7a6c224c6abc8b7a4b9c28db47)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:17.210858 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7e476-ac7a-4032-bf43-420a93dbe475 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:17.215094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 69/135] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:16 2022] GET /identity/v3/projects/5c254b7a6c224c6abc8b7a4b9c28db47 => generated 312 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:17.223384 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf1f05-e640-4e97-b014-20df92becb28 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:17.224510 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf1f05-e640-4e97-b014-20df92becb28 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:17.225531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf1f05-e640-4e97-b014-20df92becb28 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:17.228840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 70/136] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:17 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:17.234132 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf1f05-e640-4e97-b014-20df92becb28 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:17.264413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 67/137] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:50:16 2022] PUT /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/users/d0a54582166343e5baada31da5b8f7d5/roles/4498e695105347909981a684f613b6a2 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:50:17.266134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1edd-f98e-43ad-bd0a-3aabdb02ea59 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:17.266713 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1edd-f98e-43ad-bd0a-3aabdb02ea59 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:17.266941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1edd-f98e-43ad-bd0a-3aabdb02ea59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:17.290536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e6ff1edd-f98e-43ad-bd0a-3aabdb02ea59 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:17.292001 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e6ff1edd-f98e-43ad-bd0a-3aabdb02ea59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:17.356218 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:17.488402 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 68/138] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:17.507110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf1f05-e640-4e97-b014-20df92becb28 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:17.535726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:17.564344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1edd-f98e-43ad-bd0a-3aabdb02ea59 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:17.572289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160f5a9-48f8-4037-8d8a-80dc0181fa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:17.572289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160f5a9-48f8-4037-8d8a-80dc0181fa3e None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:17.572289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160f5a9-48f8-4037-8d8a-80dc0181fa3e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:17.572289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160f5a9-48f8-4037-8d8a-80dc0181fa3e None admin] PATH_INFO: `/v3/projects/5c254b7a6c224c6abc8b7a4b9c28db47/users/d0a54582166343e5baada31da5b8f7d5/roles/4498e695105347909981a684f613b6a2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:17.574228 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160f5a9-48f8-4037-8d8a-80dc0181fa3e None admin] RBAC: Authorizing `identity:create_grant(project_id=5c254b7a6c224c6abc8b7a4b9c28db47, user_id=d0a54582166343e5baada31da5b8f7d5, role_id=4498e695105347909981a684f613b6a2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:17.577599 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160f5a9-48f8-4037-8d8a-80dc0181fa3e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:17.690371 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 71/139] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:50:17 2022] PUT /identity/v3/projects/5c254b7a6c224c6abc8b7a4b9c28db47/users/d0a54582166343e5baada31da5b8f7d5/roles/4498e695105347909981a684f613b6a2 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:50:17.801341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1edd-f98e-43ad-bd0a-3aabdb02ea59 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:17.882736 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890654e-79ba-499b-9f33-d029fd38468c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:17.884938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890654e-79ba-499b-9f33-d029fd38468c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:17.885748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890654e-79ba-499b-9f33-d029fd38468c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:17.886509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890654e-79ba-499b-9f33-d029fd38468c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:17.918057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890654e-79ba-499b-9f33-d029fd38468c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-0890654e-79ba-499b-9f33-d029fd38468c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:17.938208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:17.939801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:17.941211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:17.946518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 69/140] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:50:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:17.957454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160f5a9-48f8-4037-8d8a-80dc0181fa3e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:18.115336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160f5a9-48f8-4037-8d8a-80dc0181fa3e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:18.175970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b889c-db3b-4496-9837-bb9dea364b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:18.176721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b889c-db3b-4496-9837-bb9dea364b85 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:18.177014 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b889c-db3b-4496-9837-bb9dea364b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:18.177296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b889c-db3b-4496-9837-bb9dea364b85 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:18.178066 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b889c-db3b-4496-9837-bb9dea364b85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:18.178437 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b889c-db3b-4496-9837-bb9dea364b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:18.180202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b889c-db3b-4496-9837-bb9dea364b85 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:18.190442 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 72/141] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:18.201970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890654e-79ba-499b-9f33-d029fd38468c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:18.281142 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:18.462038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890654e-79ba-499b-9f33-d029fd38468c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:18.536947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0e184-bd24-4cb2-aa75-59c0b2e06e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:18.539614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0e184-bd24-4cb2-aa75-59c0b2e06e2d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:18.540508 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0e184-bd24-4cb2-aa75-59c0b2e06e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:18.541310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0e184-bd24-4cb2-aa75-59c0b2e06e2d None admin] PATH_INFO: `/v3/users/b841e83eb2b74ca1b301a4644bbdbe93` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:18.544000 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0e184-bd24-4cb2-aa75-59c0b2e06e2d None admin] RBAC: Authorizing `identity:get_user(user_id=b841e83eb2b74ca1b301a4644bbdbe93)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:18.547339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0e184-bd24-4cb2-aa75-59c0b2e06e2d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:18.551034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 70/142] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:18 2022] GET /identity/v3/users/b841e83eb2b74ca1b301a4644bbdbe93 => generated 247 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:18.559588 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b889c-db3b-4496-9837-bb9dea364b85 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:18.644828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:18.750093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b889c-db3b-4496-9837-bb9dea364b85 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:18.801614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af8e4d-aa69-47bc-a7c0-2147fac3167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:18.802332 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af8e4d-aa69-47bc-a7c0-2147fac3167d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:18.802625 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af8e4d-aa69-47bc-a7c0-2147fac3167d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:18.803023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af8e4d-aa69-47bc-a7c0-2147fac3167d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:18.917693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af8e4d-aa69-47bc-a7c0-2147fac3167d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:18.919212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af8e4d-aa69-47bc-a7c0-2147fac3167d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:18.992771 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:19.109737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 73/143] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:19.118831 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0e184-bd24-4cb2-aa75-59c0b2e06e2d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:19.269094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2416118-d9d0-46dd-8c17-110fd369d1d3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:19.269601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2416118-d9d0-46dd-8c17-110fd369d1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:19.270051 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2416118-d9d0-46dd-8c17-110fd369d1d3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:19.271605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 74/144] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:19.295839 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a48646-acdf-4823-98f4-5fd019cf4fb2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:19.296250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a48646-acdf-4823-98f4-5fd019cf4fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:19.296556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a48646-acdf-4823-98f4-5fd019cf4fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:19.342253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-54a48646-acdf-4823-98f4-5fd019cf4fb2 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:19.348253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-54a48646-acdf-4823-98f4-5fd019cf4fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:19.373666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:19.385312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0e184-bd24-4cb2-aa75-59c0b2e06e2d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:19.461914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1e515-0c5e-4939-9590-7debf0002023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:19.466009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e515-0c5e-4939-9590-7debf0002023 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:19.466816 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e515-0c5e-4939-9590-7debf0002023 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:19.467700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e515-0c5e-4939-9590-7debf0002023 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:19.474503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1e515-0c5e-4939-9590-7debf0002023 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:19.477651 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1e515-0c5e-4939-9590-7debf0002023 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:19.480904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 71/145] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:19 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:19.490457 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1e515-0c5e-4939-9590-7debf0002023 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:19.531996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 75/146] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:19.578043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a48646-acdf-4823-98f4-5fd019cf4fb2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:19.742606 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:19.761465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1e515-0c5e-4939-9590-7debf0002023 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:19.810128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a48646-acdf-4823-98f4-5fd019cf4fb2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:19.839371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a79f83-30e2-4dbc-ac36-ab3c384276db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:19.840486 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79f83-30e2-4dbc-ac36-ab3c384276db None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:19.840486 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79f83-30e2-4dbc-ac36-ab3c384276db None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:19.840770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79f83-30e2-4dbc-ac36-ab3c384276db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:19.841536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a79f83-30e2-4dbc-ac36-ab3c384276db None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de, user.id=b841e83eb2b74ca1b301a4644bbdbe93) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:19.841888 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a79f83-30e2-4dbc-ac36-ab3c384276db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:19.843365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a79f83-30e2-4dbc-ac36-ab3c384276db None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:19.858796 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 72/147] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:19 2022] GET /identity/v3/role_assignments?user.id=b841e83eb2b74ca1b301a4644bbdbe93&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 608 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:19.860796 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e280487-e9e5-4b80-b277-0b8a28d9fc50 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:19.861293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e280487-e9e5-4b80-b277-0b8a28d9fc50 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:19.861769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e280487-e9e5-4b80-b277-0b8a28d9fc50 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:19.868635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 73/148] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:19.882569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878c8e0-4976-4c91-ab77-b75ec202d0b3 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:19.882569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878c8e0-4976-4c91-ab77-b75ec202d0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:19.882569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878c8e0-4976-4c91-ab77-b75ec202d0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:19.892353 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabd402-bd48-40e2-a2af-08418600f593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:19.893678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabd402-bd48-40e2-a2af-08418600f593 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:19.894142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabd402-bd48-40e2-a2af-08418600f593 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:19.894531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabd402-bd48-40e2-a2af-08418600f593 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:19.904439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabd402-bd48-40e2-a2af-08418600f593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:19.906818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabd402-bd48-40e2-a2af-08418600f593 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:19.909062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 76/149] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:19.915939 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabd402-bd48-40e2-a2af-08418600f593 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:19.925774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-2878c8e0-4976-4c91-ab77-b75ec202d0b3 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:19.927432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-2878c8e0-4976-4c91-ab77-b75ec202d0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:20.008602 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:20.161634 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 74/150] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:50:20.189512 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabd402-bd48-40e2-a2af-08418600f593 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:20.210291 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878c8e0-4976-4c91-ab77-b75ec202d0b3 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:20.269066 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb70ae-aa65-4800-bc6b-5b0a16a36839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:20.270316 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb70ae-aa65-4800-bc6b-5b0a16a36839 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:20.271056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb70ae-aa65-4800-bc6b-5b0a16a36839 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:20.271597 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb70ae-aa65-4800-bc6b-5b0a16a36839 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:20.273020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb70ae-aa65-4800-bc6b-5b0a16a36839 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:20.275766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb70ae-aa65-4800-bc6b-5b0a16a36839 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:20.295632 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:20.362287 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 77/151] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:50:19 2022] POST /identity/v3/users => generated 283 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:50:20.378987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878c8e0-4976-4c91-ab77-b75ec202d0b3 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:20.436003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6718fb-6ded-45e8-9f14-5083d57003d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:20.436848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6718fb-6ded-45e8-9f14-5083d57003d4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:20.437305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6718fb-6ded-45e8-9f14-5083d57003d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:20.437593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6718fb-6ded-45e8-9f14-5083d57003d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:20.456558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6718fb-6ded-45e8-9f14-5083d57003d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-9f6718fb-6ded-45e8-9f14-5083d57003d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:20.469144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:20.471245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:20.473285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 75/152] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:50:20.479177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb70ae-aa65-4800-bc6b-5b0a16a36839 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:20.656002 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb70ae-aa65-4800-bc6b-5b0a16a36839 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:20.728694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ce585-820b-41b8-b731-62196c5095ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:20.729393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ce585-820b-41b8-b731-62196c5095ee None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:20.729681 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ce585-820b-41b8-b731-62196c5095ee None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:20.729966 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ce585-820b-41b8-b731-62196c5095ee None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:20.730735 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ce585-820b-41b8-b731-62196c5095ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:20.731045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ce585-820b-41b8-b731-62196c5095ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:20.732497 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ce585-820b-41b8-b731-62196c5095ee None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:20.748255 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 78/153] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:20.753866 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6718fb-6ded-45e8-9f14-5083d57003d4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:20.793578 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:20.900042 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6718fb-6ded-45e8-9f14-5083d57003d4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:20.937314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b28957-79de-4191-a490-e050d534d040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:20.938207 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b28957-79de-4191-a490-e050d534d040 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:20.938499 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b28957-79de-4191-a490-e050d534d040 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:20.938781 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b28957-79de-4191-a490-e050d534d040 None admin] PATH_INFO: `/v3/users/369fe16545434c1fa7f21a3e742ac0b9` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:20.940034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b28957-79de-4191-a490-e050d534d040 None admin] RBAC: Authorizing `identity:get_user(user_id=369fe16545434c1fa7f21a3e742ac0b9)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:20.941539 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b28957-79de-4191-a490-e050d534d040 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:20.943468 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 76/154] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:20 2022] GET /identity/v3/users/369fe16545434c1fa7f21a3e742ac0b9 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:20.949778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ce585-820b-41b8-b731-62196c5095ee None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:21.026102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:21.078402 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682c41d-8265-4c79-af9c-dcd5dbf7da92 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:21.078586 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682c41d-8265-4c79-af9c-dcd5dbf7da92 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:21.078883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682c41d-8265-4c79-af9c-dcd5dbf7da92 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:21.080610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 77/155] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:21.088332 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3e5cc-d681-4121-9988-013ef73e112b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:21.088826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3e5cc-d681-4121-9988-013ef73e112b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:21.089310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3e5cc-d681-4121-9988-013ef73e112b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:21.097650 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ce585-820b-41b8-b731-62196c5095ee None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:21.112874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-56c3e5cc-d681-4121-9988-013ef73e112b None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:21.114240 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-56c3e5cc-d681-4121-9988-013ef73e112b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:21.151663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48deb8f-ac20-4f3e-b2e4-a7f081284767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:21.152946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48deb8f-ac20-4f3e-b2e4-a7f081284767 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:21.153436 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48deb8f-ac20-4f3e-b2e4-a7f081284767 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:21.153915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48deb8f-ac20-4f3e-b2e4-a7f081284767 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:21.156166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48deb8f-ac20-4f3e-b2e4-a7f081284767 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:21.157725 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48deb8f-ac20-4f3e-b2e4-a7f081284767 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:21.160639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 79/156] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:20 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:21.165640 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48deb8f-ac20-4f3e-b2e4-a7f081284767 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:21.221536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 78/157] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:50:21.249529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3e5cc-d681-4121-9988-013ef73e112b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:21.271067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:21.316251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48deb8f-ac20-4f3e-b2e4-a7f081284767 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:21.348386 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:21.365188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba118a-d0d8-405e-bbad-91c8997fcfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:21.366684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba118a-d0d8-405e-bbad-91c8997fcfee None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:21.367197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba118a-d0d8-405e-bbad-91c8997fcfee None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:21.367704 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba118a-d0d8-405e-bbad-91c8997fcfee None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/users/369fe16545434c1fa7f21a3e742ac0b9/roles/8e0ef0ea0d26497db73b62d43d4fda22` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:21.370276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba118a-d0d8-405e-bbad-91c8997fcfee None admin] RBAC: Authorizing `identity:create_grant(project_id=7c4956db4fa4484cbedbb6ce367d55de, user_id=369fe16545434c1fa7f21a3e742ac0b9, role_id=8e0ef0ea0d26497db73b62d43d4fda22)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:21.372204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba118a-d0d8-405e-bbad-91c8997fcfee None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:21.404056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3e5cc-d681-4121-9988-013ef73e112b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:21.452972 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 80/158] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:50:21 2022] PUT /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/users/369fe16545434c1fa7f21a3e742ac0b9/roles/8e0ef0ea0d26497db73b62d43d4fda22 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:50:21.458578 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91918a10-d89b-4e6f-901c-13b5f9159e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:21.460479 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91918a10-d89b-4e6f-901c-13b5f9159e4e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:21.461218 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91918a10-d89b-4e6f-901c-13b5f9159e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:21.461758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91918a10-d89b-4e6f-901c-13b5f9159e4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:21.483940 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91918a10-d89b-4e6f-901c-13b5f9159e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-91918a10-d89b-4e6f-901c-13b5f9159e4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:21.499412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:21.501100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:21.502434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:21.506212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 79/159] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:50:21.516046 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba118a-d0d8-405e-bbad-91c8997fcfee None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:21.660711 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba118a-d0d8-405e-bbad-91c8997fcfee None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:21.717156 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b3e4-b104-4ba8-94c4-046b996eac70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:21.718418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b3e4-b104-4ba8-94c4-046b996eac70 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:21.719019 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b3e4-b104-4ba8-94c4-046b996eac70 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:21.719704 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b3e4-b104-4ba8-94c4-046b996eac70 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:21.721366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2b3e4-b104-4ba8-94c4-046b996eac70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:21.722112 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2b3e4-b104-4ba8-94c4-046b996eac70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:21.724921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2b3e4-b104-4ba8-94c4-046b996eac70 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:21.742889 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 81/160] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:21.751721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91918a10-d89b-4e6f-901c-13b5f9159e4e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:21.866344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:21.895824 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91918a10-d89b-4e6f-901c-13b5f9159e4e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:21.950429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f133cf1-cc21-4c44-baaa-2874f43282b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:21.951914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f133cf1-cc21-4c44-baaa-2874f43282b9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:21.952697 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f133cf1-cc21-4c44-baaa-2874f43282b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:21.953351 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f133cf1-cc21-4c44-baaa-2874f43282b9 None admin] PATH_INFO: `/v3/users/d0a54582166343e5baada31da5b8f7d5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:21.955476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f133cf1-cc21-4c44-baaa-2874f43282b9 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a54582166343e5baada31da5b8f7d5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:21.958041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f133cf1-cc21-4c44-baaa-2874f43282b9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:21.962012 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 80/161] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:21 2022] GET /identity/v3/users/d0a54582166343e5baada31da5b8f7d5 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:21.966973 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b3e4-b104-4ba8-94c4-046b996eac70 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:21.982568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48cdd5-a127-4eed-b86d-72379c06d6ed None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:21.983346 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48cdd5-a127-4eed-b86d-72379c06d6ed None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:21.984111 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48cdd5-a127-4eed-b86d-72379c06d6ed None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:21.986347 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 81/162] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:21.995491 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619c76e-6135-4b60-ac3e-6957c53702fd None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:21.996334 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619c76e-6135-4b60-ac3e-6957c53702fd None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:21.997237 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619c76e-6135-4b60-ac3e-6957c53702fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:22.031009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1619c76e-6135-4b60-ac3e-6957c53702fd None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:22.033007 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1619c76e-6135-4b60-ac3e-6957c53702fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:22.065636 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:22.160421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b3e4-b104-4ba8-94c4-046b996eac70 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:22.175191 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 82/163] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:50:22.209166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20ff4f-05bd-4dda-b0f0-92784703a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:22.210986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20ff4f-05bd-4dda-b0f0-92784703a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:22.211481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20ff4f-05bd-4dda-b0f0-92784703a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:22.211959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20ff4f-05bd-4dda-b0f0-92784703a7f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:22.217709 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619c76e-6135-4b60-ac3e-6957c53702fd None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:22.413856 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20ff4f-05bd-4dda-b0f0-92784703a7f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:22.416313 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20ff4f-05bd-4dda-b0f0-92784703a7f6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:22.484703 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:22.559391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:22.595188 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619c76e-6135-4b60-ac3e-6957c53702fd None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:22.635027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 82/164] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:22.641594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20ff4f-05bd-4dda-b0f0-92784703a7f6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:22.733293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1961f8e-2708-46cb-a4af-694d04456c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:22.734571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961f8e-2708-46cb-a4af-694d04456c60 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:22.735078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961f8e-2708-46cb-a4af-694d04456c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:22.735466 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961f8e-2708-46cb-a4af-694d04456c60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:22.777224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1961f8e-2708-46cb-a4af-694d04456c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d1961f8e-2708-46cb-a4af-694d04456c60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:22.797208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:22.798306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:22.799094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:22.800786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 83/165] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:50:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 583 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:22.807898 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1961f8e-2708-46cb-a4af-694d04456c60 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:22.864448 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20ff4f-05bd-4dda-b0f0-92784703a7f6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:22.957835 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3692d2f-454c-4f00-9e67-6a48739cd6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:22.959114 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3692d2f-454c-4f00-9e67-6a48739cd6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:22.959656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3692d2f-454c-4f00-9e67-6a48739cd6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:22.960136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3692d2f-454c-4f00-9e67-6a48739cd6c2 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:22.965190 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3692d2f-454c-4f00-9e67-6a48739cd6c2 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:22.967067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3692d2f-454c-4f00-9e67-6a48739cd6c2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:22.969578 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 83/166] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:22 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:22.971149 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576c6f2-e82f-404a-ab35-20b33259ed1e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:22.971660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576c6f2-e82f-404a-ab35-20b33259ed1e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:22.972135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576c6f2-e82f-404a-ab35-20b33259ed1e None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:22.974735 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 84/167] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:22.980986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb126bf-f190-45b2-bf00-c77b46640f4e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:22.981577 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb126bf-f190-45b2-bf00-c77b46640f4e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:22.982082 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb126bf-f190-45b2-bf00-c77b46640f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:23.021330 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2fb126bf-f190-45b2-bf00-c77b46640f4e None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:23.023788 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2fb126bf-f190-45b2-bf00-c77b46640f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:23.030536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1961f8e-2708-46cb-a4af-694d04456c60 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:23.049047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:23.108550 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd40f1b-dafe-40ef-9e8e-5c7fb0dbc932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:23.109553 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd40f1b-dafe-40ef-9e8e-5c7fb0dbc932 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:23.109988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd40f1b-dafe-40ef-9e8e-5c7fb0dbc932 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:23.110401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd40f1b-dafe-40ef-9e8e-5c7fb0dbc932 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:23.111876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd40f1b-dafe-40ef-9e8e-5c7fb0dbc932 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:23.112396 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd40f1b-dafe-40ef-9e8e-5c7fb0dbc932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:23.115208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd40f1b-dafe-40ef-9e8e-5c7fb0dbc932 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:23.118235 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:23.130716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 84/168] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:50:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:23.132326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd40f1b-dafe-40ef-9e8e-5c7fb0dbc932 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:23.282514 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd40f1b-dafe-40ef-9e8e-5c7fb0dbc932 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:23.290578 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 85/169] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:23.292711 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb126bf-f190-45b2-bf00-c77b46640f4e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:23.325803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c496a0-0ff4-43be-9596-d59da94b03cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:23.326493 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c496a0-0ff4-43be-9596-d59da94b03cc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:23.326790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c496a0-0ff4-43be-9596-d59da94b03cc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:23.327079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c496a0-0ff4-43be-9596-d59da94b03cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:23.327937 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c496a0-0ff4-43be-9596-d59da94b03cc None admin] RBAC: Adding query filter params (role.id=4498e695105347909981a684f613b6a2, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de, user.id=d0a54582166343e5baada31da5b8f7d5) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:23.328282 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c496a0-0ff4-43be-9596-d59da94b03cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:23.330189 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c496a0-0ff4-43be-9596-d59da94b03cc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:23.338345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 85/170] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:23 2022] GET /identity/v3/role_assignments?user.id=d0a54582166343e5baada31da5b8f7d5&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=4498e695105347909981a684f613b6a2 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:23.345857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c496a0-0ff4-43be-9596-d59da94b03cc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:23.484955 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:23.515050 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb126bf-f190-45b2-bf00-c77b46640f4e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:23.596348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8674bb-5f35-4970-bd24-53a0d7cce422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:23.597793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8674bb-5f35-4970-bd24-53a0d7cce422 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:23.598290 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8674bb-5f35-4970-bd24-53a0d7cce422 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:23.598779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8674bb-5f35-4970-bd24-53a0d7cce422 None admin] PATH_INFO: `/v3/users/d0a54582166343e5baada31da5b8f7d5` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:23.605761 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8674bb-5f35-4970-bd24-53a0d7cce422 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a54582166343e5baada31da5b8f7d5)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:23.608588 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8674bb-5f35-4970-bd24-53a0d7cce422 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:23.611943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 86/171] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:23 2022] GET /identity/v3/users/d0a54582166343e5baada31da5b8f7d5 => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:23.624329 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8674bb-5f35-4970-bd24-53a0d7cce422 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:23.627539 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c496a0-0ff4-43be-9596-d59da94b03cc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:23.669776 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:23.753095 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c62683-553e-4950-a7da-e385e0cf7707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:23.753991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c62683-553e-4950-a7da-e385e0cf7707 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:23.754281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c62683-553e-4950-a7da-e385e0cf7707 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:23.754563 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c62683-553e-4950-a7da-e385e0cf7707 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:23.780916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c62683-553e-4950-a7da-e385e0cf7707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-98c62683-553e-4950-a7da-e385e0cf7707 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:23.789731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:23.791212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:23.792698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 86/172] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:23.796969 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c62683-553e-4950-a7da-e385e0cf7707 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:23.829960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8674bb-5f35-4970-bd24-53a0d7cce422 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:23.879710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a2983-101a-4ec0-9cf2-9ed48994e62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:23.880331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a2983-101a-4ec0-9cf2-9ed48994e62b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:23.880629 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a2983-101a-4ec0-9cf2-9ed48994e62b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:23.880925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a2983-101a-4ec0-9cf2-9ed48994e62b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:23.977491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a2983-101a-4ec0-9cf2-9ed48994e62b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:23.980504 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a2983-101a-4ec0-9cf2-9ed48994e62b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:23.999891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:24.090422 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c62683-553e-4950-a7da-e385e0cf7707 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:24.173505 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:24.236602 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74508ea5-f9f0-4fac-abcd-dcc2ae0c3065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:24.238064 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74508ea5-f9f0-4fac-abcd-dcc2ae0c3065 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:24.238760 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74508ea5-f9f0-4fac-abcd-dcc2ae0c3065 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:24.239458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74508ea5-f9f0-4fac-abcd-dcc2ae0c3065 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:24.241696 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74508ea5-f9f0-4fac-abcd-dcc2ae0c3065 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:24.242758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74508ea5-f9f0-4fac-abcd-dcc2ae0c3065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:24.245603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74508ea5-f9f0-4fac-abcd-dcc2ae0c3065 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:24.281708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 87/173] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:24.291590 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74508ea5-f9f0-4fac-abcd-dcc2ae0c3065 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:24.316996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 87/174] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:24.326484 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a2983-101a-4ec0-9cf2-9ed48994e62b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:24.661286 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74508ea5-f9f0-4fac-abcd-dcc2ae0c3065 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:24.682340 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a2983-101a-4ec0-9cf2-9ed48994e62b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:24.700718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:24.730109 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68ed60-3eaa-4e6e-83ef-2205a6085afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:24.731027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68ed60-3eaa-4e6e-83ef-2205a6085afe None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:24.731315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68ed60-3eaa-4e6e-83ef-2205a6085afe None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:24.731600 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68ed60-3eaa-4e6e-83ef-2205a6085afe None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:24.733123 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68ed60-3eaa-4e6e-83ef-2205a6085afe None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:24.734837 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68ed60-3eaa-4e6e-83ef-2205a6085afe None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:24.736603 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 88/175] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:24 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:24.743070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68ed60-3eaa-4e6e-83ef-2205a6085afe None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:24.765593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e527a3-5f7b-46fb-bd82-e8776c589621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:24.766502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e527a3-5f7b-46fb-bd82-e8776c589621 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:24.766788 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e527a3-5f7b-46fb-bd82-e8776c589621 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:24.767212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e527a3-5f7b-46fb-bd82-e8776c589621 None admin] PATH_INFO: `/v3/users/d0a54582166343e5baada31da5b8f7d5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:24.773312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e527a3-5f7b-46fb-bd82-e8776c589621 None admin] RBAC: Authorizing `identity:get_user(user_id=d0a54582166343e5baada31da5b8f7d5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:24.774730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e527a3-5f7b-46fb-bd82-e8776c589621 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:24.776771 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 88/176] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:24 2022] GET /identity/v3/users/d0a54582166343e5baada31da5b8f7d5 => generated 275 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:24.784654 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e527a3-5f7b-46fb-bd82-e8776c589621 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:24.873602 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:24.902236 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68ed60-3eaa-4e6e-83ef-2205a6085afe None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:24.966390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e527a3-5f7b-46fb-bd82-e8776c589621 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:24.970050 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac722-b3b7-48be-bdcc-20cf6c591f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:24.970733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac722-b3b7-48be-bdcc-20cf6c591f2b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:24.971039 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac722-b3b7-48be-bdcc-20cf6c591f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:24.971285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac722-b3b7-48be-bdcc-20cf6c591f2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:24.972276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ac722-b3b7-48be-bdcc-20cf6c591f2b None admin] RBAC: Adding query filter params (role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de, user.id=d0a54582166343e5baada31da5b8f7d5) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:24.972499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ac722-b3b7-48be-bdcc-20cf6c591f2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:24.974140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ac722-b3b7-48be-bdcc-20cf6c591f2b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:24.987725 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 89/177] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:24 2022] GET /identity/v3/role_assignments?user.id=d0a54582166343e5baada31da5b8f7d5&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:25.035888 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188ebb1-a1f5-4c59-9e7d-70dc9bb9b0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:25.036610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ebb1-a1f5-4c59-9e7d-70dc9bb9b0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:25.036915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ebb1-a1f5-4c59-9e7d-70dc9bb9b0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:25.037190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ebb1-a1f5-4c59-9e7d-70dc9bb9b0f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:25.146444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188ebb1-a1f5-4c59-9e7d-70dc9bb9b0f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:25.147980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188ebb1-a1f5-4c59-9e7d-70dc9bb9b0f5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:25.238517 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:25.360650 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 89/178] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:50:25.369693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac722-b3b7-48be-bdcc-20cf6c591f2b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:25.434945 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:25.523988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac722-b3b7-48be-bdcc-20cf6c591f2b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:25.578882 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb64e22-adaf-4370-ad83-d1d91f7f30c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:25.580196 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb64e22-adaf-4370-ad83-d1d91f7f30c9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:25.580634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb64e22-adaf-4370-ad83-d1d91f7f30c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:25.581034 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb64e22-adaf-4370-ad83-d1d91f7f30c9 None admin] PATH_INFO: `/v3/projects/5c254b7a6c224c6abc8b7a4b9c28db47` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:25.586067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb64e22-adaf-4370-ad83-d1d91f7f30c9 None admin] RBAC: Authorizing `identity:get_project(project_id=5c254b7a6c224c6abc8b7a4b9c28db47)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:25.588326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb64e22-adaf-4370-ad83-d1d91f7f30c9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:25.591169 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 90/179] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:25 2022] GET /identity/v3/projects/5c254b7a6c224c6abc8b7a4b9c28db47 => generated 312 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:25.601706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188ebb1-a1f5-4c59-9e7d-70dc9bb9b0f5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:25.858867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188ebb1-a1f5-4c59-9e7d-70dc9bb9b0f5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:25.862395 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d827fea-9777-452d-a2c3-a693010b0c86 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:25.862693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d827fea-9777-452d-a2c3-a693010b0c86 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:25.862979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d827fea-9777-452d-a2c3-a693010b0c86 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:25.866017 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 91/180] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:25.874563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b70dc-fb8e-4a5d-89b7-53ff4d8e084a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:25.874856 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b70dc-fb8e-4a5d-89b7-53ff4d8e084a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:25.875154 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b70dc-fb8e-4a5d-89b7-53ff4d8e084a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:25.917514 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e59b70dc-fb8e-4a5d-89b7-53ff4d8e084a None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:25.919410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e59b70dc-fb8e-4a5d-89b7-53ff4d8e084a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:25.945144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7276fb-4a39-4e84-a617-eac1add79ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:25.946245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7276fb-4a39-4e84-a617-eac1add79ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:25.946757 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7276fb-4a39-4e84-a617-eac1add79ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:25.947275 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7276fb-4a39-4e84-a617-eac1add79ea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:25.948922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7276fb-4a39-4e84-a617-eac1add79ea5 None admin] RBAC: Adding query filter params (role.id=4498e695105347909981a684f613b6a2, scope.project.id=5c254b7a6c224c6abc8b7a4b9c28db47, user.id=d0a54582166343e5baada31da5b8f7d5) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:25.949495 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7276fb-4a39-4e84-a617-eac1add79ea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:25.954906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7276fb-4a39-4e84-a617-eac1add79ea5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:25.967729 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 90/181] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:25 2022] GET /identity/v3/role_assignments?user.id=d0a54582166343e5baada31da5b8f7d5&scope.project.id=5c254b7a6c224c6abc8b7a4b9c28db47&role.id=4498e695105347909981a684f613b6a2 => generated 608 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:26.057811 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 92/182] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:26.101404 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7276fb-4a39-4e84-a617-eac1add79ea5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:26.134740 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:26.250177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2865021-ddf3-461d-9f32-e04e6d0859e7 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.250853 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2865021-ddf3-461d-9f32-e04e6d0859e7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.251353 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2865021-ddf3-461d-9f32-e04e6d0859e7 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.253144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 93/183] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:26.261011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ba21f-b451-4bfe-99de-9fee49718d9c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.261665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ba21f-b451-4bfe-99de-9fee49718d9c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.262157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ba21f-b451-4bfe-99de-9fee49718d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.293009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2c2ba21f-b451-4bfe-99de-9fee49718d9c None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:26.295787 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2c2ba21f-b451-4bfe-99de-9fee49718d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:26.302758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7276fb-4a39-4e84-a617-eac1add79ea5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:26.340648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:26.369316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca2512-4dc1-4cc1-ba2b-83db541da18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:26.370140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca2512-4dc1-4cc1-ba2b-83db541da18a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.370424 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca2512-4dc1-4cc1-ba2b-83db541da18a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.370706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca2512-4dc1-4cc1-ba2b-83db541da18a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.387782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca2512-4dc1-4cc1-ba2b-83db541da18a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2dca2512-4dc1-4cc1-ba2b-83db541da18a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:26.395003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:26.396270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 91/184] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:26.400490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a3a98-0666-46f5-a8f0-3975eeafee3f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.401221 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a3a98-0666-46f5-a8f0-3975eeafee3f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.401221 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a3a98-0666-46f5-a8f0-3975eeafee3f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.402846 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 92/185] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:26.410955 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ba0c4-3b06-4dee-8053-1277de26689d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.411397 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ba0c4-3b06-4dee-8053-1277de26689d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.411831 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ba0c4-3b06-4dee-8053-1277de26689d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.420491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 94/186] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:26.421778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ba21f-b451-4bfe-99de-9fee49718d9c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:26.439190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ad8ba0c4-3b06-4dee-8053-1277de26689d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:26.441100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ad8ba0c4-3b06-4dee-8053-1277de26689d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:26.556575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 93/187] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:26.564524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3260770-0dd4-4675-ae58-1e53953551f0 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.564524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3260770-0dd4-4675-ae58-1e53953551f0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.564524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3260770-0dd4-4675-ae58-1e53953551f0 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.564524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 94/188] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:26.564524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3260770-0dd4-4675-ae58-1e53953551f0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:26.569962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:26.596890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ba21f-b451-4bfe-99de-9fee49718d9c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:26.656013 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeca1c-581d-4685-8df6-b8eacb9f375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:26.657129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeca1c-581d-4685-8df6-b8eacb9f375d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.657549 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeca1c-581d-4685-8df6-b8eacb9f375d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.657952 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeca1c-581d-4685-8df6-b8eacb9f375d None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.659102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeca1c-581d-4685-8df6-b8eacb9f375d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:26.659561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeca1c-581d-4685-8df6-b8eacb9f375d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:26.661495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeca1c-581d-4685-8df6-b8eacb9f375d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:26.676318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 95/189] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:26.679531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c95f31-9067-475c-87ef-5e68d15e9000 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.680029 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c95f31-9067-475c-87ef-5e68d15e9000 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.680544 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c95f31-9067-475c-87ef-5e68d15e9000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.708286 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-21c95f31-9067-475c-87ef-5e68d15e9000 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:26.710851 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-21c95f31-9067-475c-87ef-5e68d15e9000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:26.715534 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3260770-0dd4-4675-ae58-1e53953551f0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:26.753252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:26.763757 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974334a3-9d7c-4f74-8e52-fd029674e4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:26.765067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974334a3-9d7c-4f74-8e52-fd029674e4cf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.765531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974334a3-9d7c-4f74-8e52-fd029674e4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.766222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974334a3-9d7c-4f74-8e52-fd029674e4cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.781765 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974334a3-9d7c-4f74-8e52-fd029674e4cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-974334a3-9d7c-4f74-8e52-fd029674e4cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:26.790433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:26.792052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 95/190] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:26.793558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974334a3-9d7c-4f74-8e52-fd029674e4cf None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:26.810733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 96/191] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:26.813710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c95f31-9067-475c-87ef-5e68d15e9000 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:26.863606 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:26.923204 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974334a3-9d7c-4f74-8e52-fd029674e4cf None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:26.968441 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070573a-5e55-4696-ae7d-b4a06a397aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:26.970542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c95f31-9067-475c-87ef-5e68d15e9000 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:26.970691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070573a-5e55-4696-ae7d-b4a06a397aec None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.970691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070573a-5e55-4696-ae7d-b4a06a397aec None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.970691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070573a-5e55-4696-ae7d-b4a06a397aec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.989190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070573a-5e55-4696-ae7d-b4a06a397aec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4070573a-5e55-4696-ae7d-b4a06a397aec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:27.003725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:27.005492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 96/192] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:50:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:27.007088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070573a-5e55-4696-ae7d-b4a06a397aec None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:27.031432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a99d9-0a9d-463c-b7ad-13af2a9f63f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:27.032519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a99d9-0a9d-463c-b7ad-13af2a9f63f2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:27.032868 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a99d9-0a9d-463c-b7ad-13af2a9f63f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:27.033221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a99d9-0a9d-463c-b7ad-13af2a9f63f2 None admin] PATH_INFO: `/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:27.058073 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a99d9-0a9d-463c-b7ad-13af2a9f63f2 None admin] RBAC: Authorizing `identity:get_user(user_id=07f5fc86e79c434b9bebe6bbcfdd14b5)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:27.059583 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a99d9-0a9d-463c-b7ad-13af2a9f63f2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:27.061487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 97/193] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:26 2022] GET /identity/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5 => generated 283 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:27.062258 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a99d9-0a9d-463c-b7ad-13af2a9f63f2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:27.096814 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:27.157584 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070573a-5e55-4696-ae7d-b4a06a397aec None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:27.181596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:27.192945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b0f44-7649-436f-8907-7a1e3731339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:27.194340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b0f44-7649-436f-8907-7a1e3731339e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:27.194929 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b0f44-7649-436f-8907-7a1e3731339e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:27.195253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b0f44-7649-436f-8907-7a1e3731339e None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:27.196108 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b0f44-7649-436f-8907-7a1e3731339e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:27.196556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b0f44-7649-436f-8907-7a1e3731339e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:27.198341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b0f44-7649-436f-8907-7a1e3731339e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:27.200081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a99d9-0a9d-463c-b7ad-13af2a9f63f2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:27.213252 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 97/194] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:27.214502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b0f44-7649-436f-8907-7a1e3731339e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:27.246332 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a3946-edbc-4bf3-894f-6030ef22af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:27.247132 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3946-edbc-4bf3-894f-6030ef22af05 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:27.247421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3946-edbc-4bf3-894f-6030ef22af05 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:27.247703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3946-edbc-4bf3-894f-6030ef22af05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:27.248932 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a3946-edbc-4bf3-894f-6030ef22af05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:27.250387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a3946-edbc-4bf3-894f-6030ef22af05 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:27.259064 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 98/195] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:27.259064 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a3946-edbc-4bf3-894f-6030ef22af05 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:27.354088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b0f44-7649-436f-8907-7a1e3731339e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:27.376890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:27.390169 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a3946-edbc-4bf3-894f-6030ef22af05 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:27.402929 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55717f-f8fe-4eee-8063-76e0887e8103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:27.404005 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55717f-f8fe-4eee-8063-76e0887e8103 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:27.404488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55717f-f8fe-4eee-8063-76e0887e8103 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:27.404864 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55717f-f8fe-4eee-8063-76e0887e8103 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:27.406022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55717f-f8fe-4eee-8063-76e0887e8103 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:27.406596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55717f-f8fe-4eee-8063-76e0887e8103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:27.408678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55717f-f8fe-4eee-8063-76e0887e8103 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:27.418033 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 98/196] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:50:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:27.418950 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55717f-f8fe-4eee-8063-76e0887e8103 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:27.473315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337ad39-0b36-4c91-bf97-5952171927ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:27.473315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337ad39-0b36-4c91-bf97-5952171927ed None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:27.473315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337ad39-0b36-4c91-bf97-5952171927ed None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:27.473315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337ad39-0b36-4c91-bf97-5952171927ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:27.495230 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:27.538709 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337ad39-0b36-4c91-bf97-5952171927ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:27.541487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337ad39-0b36-4c91-bf97-5952171927ed None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:27.562426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55717f-f8fe-4eee-8063-76e0887e8103 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:27.603255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15a220-46fb-46b0-b572-39985b8f93dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:27.604658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a220-46fb-46b0-b572-39985b8f93dc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:27.605054 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a220-46fb-46b0-b572-39985b8f93dc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:27.605447 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a220-46fb-46b0-b572-39985b8f93dc None admin] PATH_INFO: `/v3/users/369fe16545434c1fa7f21a3e742ac0b9` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:27.607407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15a220-46fb-46b0-b572-39985b8f93dc None admin] RBAC: Authorizing `identity:get_user(user_id=369fe16545434c1fa7f21a3e742ac0b9)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:27.609596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15a220-46fb-46b0-b572-39985b8f93dc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:27.620442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 99/197] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:27 2022] GET /identity/v3/users/369fe16545434c1fa7f21a3e742ac0b9 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:27.620442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15a220-46fb-46b0-b572-39985b8f93dc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:27.668749 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:27.689575 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 99/198] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:27.691077 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337ad39-0b36-4c91-bf97-5952171927ed None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:27.755622 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15a220-46fb-46b0-b572-39985b8f93dc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:27.790860 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:27.795968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264f2d5-f205-48ea-86ad-ab520992c1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:27.796693 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264f2d5-f205-48ea-86ad-ab520992c1f9 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:27.796980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264f2d5-f205-48ea-86ad-ab520992c1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:27.797263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264f2d5-f205-48ea-86ad-ab520992c1f9 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:27.798093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264f2d5-f205-48ea-86ad-ab520992c1f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:27.799598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264f2d5-f205-48ea-86ad-ab520992c1f9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:27.819971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337ad39-0b36-4c91-bf97-5952171927ed None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:27.857365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818541-9399-4ca1-b8b1-2869842ae7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:27.858290 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818541-9399-4ca1-b8b1-2869842ae7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:27.858633 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818541-9399-4ca1-b8b1-2869842ae7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:27.858911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818541-9399-4ca1-b8b1-2869842ae7a4 None admin] PATH_INFO: `/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:27.860104 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97818541-9399-4ca1-b8b1-2869842ae7a4 None admin] RBAC: Authorizing `identity:get_user(user_id=07f5fc86e79c434b9bebe6bbcfdd14b5)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:27.861596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97818541-9399-4ca1-b8b1-2869842ae7a4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:27.863306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 100/199] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:27 2022] GET /identity/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:27.864051 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818541-9399-4ca1-b8b1-2869842ae7a4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:27.941158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:27.994321 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818541-9399-4ca1-b8b1-2869842ae7a4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:28.035236 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38ed55-8805-42cf-b3c0-9996ae48e650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:28.035892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38ed55-8805-42cf-b3c0-9996ae48e650 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:28.036184 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38ed55-8805-42cf-b3c0-9996ae48e650 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:28.036494 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38ed55-8805-42cf-b3c0-9996ae48e650 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:28.038233 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 100/200] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:50:27 2022] POST /identity/v3/users => generated 297 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:50:28.053007 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264f2d5-f205-48ea-86ad-ab520992c1f9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:28.149228 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38ed55-8805-42cf-b3c0-9996ae48e650 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:28.151718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38ed55-8805-42cf-b3c0-9996ae48e650 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:28.200516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264f2d5-f205-48ea-86ad-ab520992c1f9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:28.251858 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:28.255220 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b90a3-30ac-4504-b4a8-526d05b741cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:28.258081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b90a3-30ac-4504-b4a8-526d05b741cc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:28.258439 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b90a3-30ac-4504-b4a8-526d05b741cc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:28.259139 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b90a3-30ac-4504-b4a8-526d05b741cc None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:28.260892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b90a3-30ac-4504-b4a8-526d05b741cc None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:28.263121 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b90a3-30ac-4504-b4a8-526d05b741cc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:28.266833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 101/201] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:28 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:28.266833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b90a3-30ac-4504-b4a8-526d05b741cc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:28.312834 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 101/202] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:28.314682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38ed55-8805-42cf-b3c0-9996ae48e650 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:28.346762 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:28.445758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b90a3-30ac-4504-b4a8-526d05b741cc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:28.489646 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38ed55-8805-42cf-b3c0-9996ae48e650 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:28.506654 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e111325-9534-4b13-8e58-c4328105a3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:28.508041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e111325-9534-4b13-8e58-c4328105a3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:28.508525 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e111325-9534-4b13-8e58-c4328105a3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:28.509608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e111325-9534-4b13-8e58-c4328105a3e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:28.549797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b513828-8003-4eb2-894e-82de32e5fe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:28.550569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b513828-8003-4eb2-894e-82de32e5fe8f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:28.551082 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b513828-8003-4eb2-894e-82de32e5fe8f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:28.551612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b513828-8003-4eb2-894e-82de32e5fe8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:28.553817 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b513828-8003-4eb2-894e-82de32e5fe8f None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=5589046ca8fb418da62e5cef63cedc9f, user.id=07f5fc86e79c434b9bebe6bbcfdd14b5) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:28.554392 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b513828-8003-4eb2-894e-82de32e5fe8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:28.557204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b513828-8003-4eb2-894e-82de32e5fe8f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:28.572908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 102/203] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:28 2022] GET /identity/v3/role_assignments?user.id=07f5fc86e79c434b9bebe6bbcfdd14b5&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:28.572908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b513828-8003-4eb2-894e-82de32e5fe8f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:28.620872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:28.622066 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e111325-9534-4b13-8e58-c4328105a3e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:28.623576 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e111325-9534-4b13-8e58-c4328105a3e6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:28.753201 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:28.798505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b513828-8003-4eb2-894e-82de32e5fe8f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:28.868467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f027746-6fb9-4baa-81d5-9ed535459d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:28.874011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f027746-6fb9-4baa-81d5-9ed535459d14 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:28.874343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f027746-6fb9-4baa-81d5-9ed535459d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:28.874629 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f027746-6fb9-4baa-81d5-9ed535459d14 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:28.876286 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f027746-6fb9-4baa-81d5-9ed535459d14 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:28.878154 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f027746-6fb9-4baa-81d5-9ed535459d14 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:28.879922 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 103/204] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:28 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:28.888279 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f027746-6fb9-4baa-81d5-9ed535459d14 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:28.952993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 102/205] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:28.969440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e111325-9534-4b13-8e58-c4328105a3e6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:29.073537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:29.087153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f027746-6fb9-4baa-81d5-9ed535459d14 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:29.153913 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5511a97-3c4e-4ddd-becf-a3dd70fdf2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:29.154939 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5511a97-3c4e-4ddd-becf-a3dd70fdf2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:29.155475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5511a97-3c4e-4ddd-becf-a3dd70fdf2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:29.155977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5511a97-3c4e-4ddd-becf-a3dd70fdf2f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:29.157631 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5511a97-3c4e-4ddd-becf-a3dd70fdf2f5 None admin] RBAC: Adding query filter params (role.id=8e0ef0ea0d26497db73b62d43d4fda22, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de, user.id=369fe16545434c1fa7f21a3e742ac0b9) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:29.158216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5511a97-3c4e-4ddd-becf-a3dd70fdf2f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:29.160690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5511a97-3c4e-4ddd-becf-a3dd70fdf2f5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:29.186669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 104/206] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:28 2022] GET /identity/v3/role_assignments?user.id=369fe16545434c1fa7f21a3e742ac0b9&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=8e0ef0ea0d26497db73b62d43d4fda22 => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:29.229564 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e111325-9534-4b13-8e58-c4328105a3e6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:29.284407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fd1ee-7a6e-40d1-b4f3-8b6deaba6660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:29.285301 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fd1ee-7a6e-40d1-b4f3-8b6deaba6660 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:29.285596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fd1ee-7a6e-40d1-b4f3-8b6deaba6660 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:29.285877 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fd1ee-7a6e-40d1-b4f3-8b6deaba6660 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:29.287265 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fd1ee-7a6e-40d1-b4f3-8b6deaba6660 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:29.289712 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fd1ee-7a6e-40d1-b4f3-8b6deaba6660 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:29.292849 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 103/207] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:28 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:29.297942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5511a97-3c4e-4ddd-becf-a3dd70fdf2f5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:29.481957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5511a97-3c4e-4ddd-becf-a3dd70fdf2f5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:29.536183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2525-28fe-4ee5-89d9-e4c625ffd313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:29.537006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2525-28fe-4ee5-89d9-e4c625ffd313 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:29.537912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2525-28fe-4ee5-89d9-e4c625ffd313 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:29.538246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2525-28fe-4ee5-89d9-e4c625ffd313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:29.539315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2525-28fe-4ee5-89d9-e4c625ffd313 None admin] RBAC: Adding query filter params (role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=5589046ca8fb418da62e5cef63cedc9f, user.id=07f5fc86e79c434b9bebe6bbcfdd14b5) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:29.539634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2525-28fe-4ee5-89d9-e4c625ffd313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:29.541303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2525-28fe-4ee5-89d9-e4c625ffd313 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:29.556936 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 105/208] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:29 2022] GET /identity/v3/role_assignments?user.id=07f5fc86e79c434b9bebe6bbcfdd14b5&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:31.079542 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610831e6-3527-41d2-beff-b2637ef15842 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.080120 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610831e6-3527-41d2-beff-b2637ef15842 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.080120 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610831e6-3527-41d2-beff-b2637ef15842 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.081633 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 104/209] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:31.087971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce84b5e-1b23-4a4e-a3ec-d3f327cd3325 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.088554 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce84b5e-1b23-4a4e-a3ec-d3f327cd3325 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.088554 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce84b5e-1b23-4a4e-a3ec-d3f327cd3325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.091255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d0617-bb16-4e92-b859-77b6273a3659 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.091643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d0617-bb16-4e92-b859-77b6273a3659 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.091879 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d0617-bb16-4e92-b859-77b6273a3659 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.093632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 105/210] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:31.104869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a0eb9-c0ef-4a04-a802-37bc2273beca None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.106163 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a0eb9-c0ef-4a04-a802-37bc2273beca None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.106706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a0eb9-c0ef-4a04-a802-37bc2273beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.112862 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fce84b5e-1b23-4a4e-a3ec-d3f327cd3325 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:31.114095 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fce84b5e-1b23-4a4e-a3ec-d3f327cd3325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:31.128147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:31.142218 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e20a0eb9-c0ef-4a04-a802-37bc2273beca None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:31.146383 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e20a0eb9-c0ef-4a04-a802-37bc2273beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:31.149957 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:31.214203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 106/211] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:31.249474 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce84b5e-1b23-4a4e-a3ec-d3f327cd3325 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:31.269587 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 106/212] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:31.274536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b5a2d-df47-4d4b-845c-a6edb432bbfe None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.274536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b5a2d-df47-4d4b-845c-a6edb432bbfe None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.274536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b5a2d-df47-4d4b-845c-a6edb432bbfe None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.275753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 107/213] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:31.284953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab698a0-0e59-4a35-915b-7a5f7f59884f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.285246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab698a0-0e59-4a35-915b-7a5f7f59884f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.285531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab698a0-0e59-4a35-915b-7a5f7f59884f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.308841 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9ab698a0-0e59-4a35-915b-7a5f7f59884f None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:31.310059 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9ab698a0-0e59-4a35-915b-7a5f7f59884f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:31.416380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 108/214] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:31.418298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab698a0-0e59-4a35-915b-7a5f7f59884f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:31.426985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce84b5e-1b23-4a4e-a3ec-d3f327cd3325 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:31.498388 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97603c-6254-4db8-adf3-73d3bd78e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:31.499686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97603c-6254-4db8-adf3-73d3bd78e67a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.500206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97603c-6254-4db8-adf3-73d3bd78e67a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.500744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97603c-6254-4db8-adf3-73d3bd78e67a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.522601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97603c-6254-4db8-adf3-73d3bd78e67a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:31.530758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-0f97603c-6254-4db8-adf3-73d3bd78e67a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:31.537209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:31.539075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 107/215] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:50:31.543319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97603c-6254-4db8-adf3-73d3bd78e67a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:31.557202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:31.594801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab698a0-0e59-4a35-915b-7a5f7f59884f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:31.654338 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129b4dc-0a1b-449e-94d6-345f8f8d5230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:31.655156 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129b4dc-0a1b-449e-94d6-345f8f8d5230 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.655450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129b4dc-0a1b-449e-94d6-345f8f8d5230 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.655854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129b4dc-0a1b-449e-94d6-345f8f8d5230 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.657884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129b4dc-0a1b-449e-94d6-345f8f8d5230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:31.660011 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129b4dc-0a1b-449e-94d6-345f8f8d5230 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:31.663048 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 109/216] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:31.664029 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129b4dc-0a1b-449e-94d6-345f8f8d5230 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:31.730271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97603c-6254-4db8-adf3-73d3bd78e67a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:31.796175 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4f64a-6316-4800-a8c7-7ef5237b08e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:31.797013 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4f64a-6316-4800-a8c7-7ef5237b08e5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.797319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4f64a-6316-4800-a8c7-7ef5237b08e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.797595 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4f64a-6316-4800-a8c7-7ef5237b08e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.816705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4f64a-6316-4800-a8c7-7ef5237b08e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:31.825963 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129b4dc-0a1b-449e-94d6-345f8f8d5230 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-6aa4f64a-6316-4800-a8c7-7ef5237b08e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:31.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:31.832153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:31.833009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:31.834499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 108/217] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:31.836302 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4f64a-6316-4800-a8c7-7ef5237b08e5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:31.886976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fed9c-de1d-4ed5-98d6-8226a2abd6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:31.888335 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fed9c-de1d-4ed5-98d6-8226a2abd6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.889139 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fed9c-de1d-4ed5-98d6-8226a2abd6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.889797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fed9c-de1d-4ed5-98d6-8226a2abd6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.891760 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fed9c-de1d-4ed5-98d6-8226a2abd6a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:31.892591 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fed9c-de1d-4ed5-98d6-8226a2abd6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:31.895722 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fed9c-de1d-4ed5-98d6-8226a2abd6a9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:31.899726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:31.909203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 110/218] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:31.910277 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fed9c-de1d-4ed5-98d6-8226a2abd6a9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:31.935285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:31.999482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4f64a-6316-4800-a8c7-7ef5237b08e5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:32.058964 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c2b7c-21c4-4ff7-ac51-45046a5b56c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:32.060942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c2b7c-21c4-4ff7-ac51-45046a5b56c4 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:32.061604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c2b7c-21c4-4ff7-ac51-45046a5b56c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:32.061975 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c2b7c-21c4-4ff7-ac51-45046a5b56c4 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:32.063420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c2b7c-21c4-4ff7-ac51-45046a5b56c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:32.065884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c2b7c-21c4-4ff7-ac51-45046a5b56c4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:32.081979 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fed9c-de1d-4ed5-98d6-8226a2abd6a9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:32.122336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b85b1-239a-42f2-a2a7-091106d2a16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:32.123029 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b85b1-239a-42f2-a2a7-091106d2a16b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:32.123322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b85b1-239a-42f2-a2a7-091106d2a16b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:32.123608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b85b1-239a-42f2-a2a7-091106d2a16b None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:32.124417 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b85b1-239a-42f2-a2a7-091106d2a16b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:32.124733 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b85b1-239a-42f2-a2a7-091106d2a16b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:32.126174 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b85b1-239a-42f2-a2a7-091106d2a16b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:32.136356 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 111/219] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:32.137273 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b85b1-239a-42f2-a2a7-091106d2a16b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:32.210089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:32.217508 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 109/220] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:50:31 2022] POST /identity/v3/users => generated 297 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:50:32.219224 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c2b7c-21c4-4ff7-ac51-45046a5b56c4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:32.273050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b85b1-239a-42f2-a2a7-091106d2a16b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:32.328637 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bda426-1a96-4094-9d00-48a3569da9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:32.329992 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bda426-1a96-4094-9d00-48a3569da9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:32.330476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bda426-1a96-4094-9d00-48a3569da9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:32.330975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bda426-1a96-4094-9d00-48a3569da9b4 None admin] PATH_INFO: `/v3/users/5055afe87d8d489b839cf2e35647edc6` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:32.356326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bda426-1a96-4094-9d00-48a3569da9b4 None admin] RBAC: Authorizing `identity:get_user(user_id=5055afe87d8d489b839cf2e35647edc6)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:32.358283 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bda426-1a96-4094-9d00-48a3569da9b4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:32.361033 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 112/221] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:32 2022] GET /identity/v3/users/5055afe87d8d489b839cf2e35647edc6 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:32.369662 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bda426-1a96-4094-9d00-48a3569da9b4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:32.419070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c2b7c-21c4-4ff7-ac51-45046a5b56c4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:32.446457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:32.463620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5855ac0-ee35-4055-972d-b3d2f5bc6062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:32.466267 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855ac0-ee35-4055-972d-b3d2f5bc6062 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:32.466267 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855ac0-ee35-4055-972d-b3d2f5bc6062 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:32.471808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5855ac0-ee35-4055-972d-b3d2f5bc6062 None admin] PATH_INFO: `/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:32.471808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5855ac0-ee35-4055-972d-b3d2f5bc6062 None admin] RBAC: Authorizing `identity:get_user(user_id=07f5fc86e79c434b9bebe6bbcfdd14b5)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:32.473076 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5855ac0-ee35-4055-972d-b3d2f5bc6062 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:32.476451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 110/222] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:32 2022] GET /identity/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:32.478847 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f89a0-3666-49e7-b98a-0ec1f5121f68 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:32.479305 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f89a0-3666-49e7-b98a-0ec1f5121f68 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:32.481020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f89a0-3666-49e7-b98a-0ec1f5121f68 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:32.488275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 111/223] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:32.488948 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f89a0-3666-49e7-b98a-0ec1f5121f68 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:32.527502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bda426-1a96-4094-9d00-48a3569da9b4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:32.550634 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:32.582741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620907f0-aa5d-45ce-ba87-7f1c8452c3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:32.584768 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620907f0-aa5d-45ce-ba87-7f1c8452c3bf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:32.585245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620907f0-aa5d-45ce-ba87-7f1c8452c3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:32.585704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620907f0-aa5d-45ce-ba87-7f1c8452c3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:32.683146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f89a0-3666-49e7-b98a-0ec1f5121f68 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:32.702357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620907f0-aa5d-45ce-ba87-7f1c8452c3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:32.705103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620907f0-aa5d-45ce-ba87-7f1c8452c3bf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:32.747269 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ed838-e2a9-4809-b994-fb42e0430f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:32.748940 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed838-e2a9-4809-b994-fb42e0430f51 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:32.749387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed838-e2a9-4809-b994-fb42e0430f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:32.749831 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed838-e2a9-4809-b994-fb42e0430f51 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:32.752322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ed838-e2a9-4809-b994-fb42e0430f51 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:32.755079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ed838-e2a9-4809-b994-fb42e0430f51 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:32.759061 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 112/224] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:32 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:32.761490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc68204-23bd-4637-a477-4ec555cf6904 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:32.761969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc68204-23bd-4637-a477-4ec555cf6904 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:32.762517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc68204-23bd-4637-a477-4ec555cf6904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:32.791369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6cc68204-23bd-4637-a477-4ec555cf6904 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:32.792941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6cc68204-23bd-4637-a477-4ec555cf6904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:32.869915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 113/225] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:32.871358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620907f0-aa5d-45ce-ba87-7f1c8452c3bf None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:32.904156 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:32.907207 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:32.937288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 113/226] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:32.939402 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc68204-23bd-4637-a477-4ec555cf6904 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:33.043467 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620907f0-aa5d-45ce-ba87-7f1c8452c3bf None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:33.107465 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc68204-23bd-4637-a477-4ec555cf6904 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:33.109655 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744fbd5-383a-40cb-89eb-769b73b5d5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:33.112549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744fbd5-383a-40cb-89eb-769b73b5d5b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:33.113018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744fbd5-383a-40cb-89eb-769b73b5d5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:33.113471 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744fbd5-383a-40cb-89eb-769b73b5d5b3 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f/users/07f5fc86e79c434b9bebe6bbcfdd14b5/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:33.118389 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744fbd5-383a-40cb-89eb-769b73b5d5b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5589046ca8fb418da62e5cef63cedc9f, user_id=07f5fc86e79c434b9bebe6bbcfdd14b5, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:33.122106 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744fbd5-383a-40cb-89eb-769b73b5d5b3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:33.151558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadb1e4-dd44-4ebf-b9f0-f8b09fd3fd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:33.152620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadb1e4-dd44-4ebf-b9f0-f8b09fd3fd75 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:33.152985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadb1e4-dd44-4ebf-b9f0-f8b09fd3fd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:33.153352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadb1e4-dd44-4ebf-b9f0-f8b09fd3fd75 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:33.154750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edadb1e4-dd44-4ebf-b9f0-f8b09fd3fd75 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:33.156471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edadb1e4-dd44-4ebf-b9f0-f8b09fd3fd75 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:33.158344 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 114/227] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:32 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:33.159237 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadb1e4-dd44-4ebf-b9f0-f8b09fd3fd75 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:33.231116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:33.243391 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 114/228] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:50:32 2022] PUT /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f/users/07f5fc86e79c434b9bebe6bbcfdd14b5/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:50:33.244737 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744fbd5-383a-40cb-89eb-769b73b5d5b3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:33.364773 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadb1e4-dd44-4ebf-b9f0-f8b09fd3fd75 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:33.428076 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cf7cb-5434-4b53-a7d5-09e99d974e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:33.429214 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cf7cb-5434-4b53-a7d5-09e99d974e45 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:33.430711 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cf7cb-5434-4b53-a7d5-09e99d974e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:33.431178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cf7cb-5434-4b53-a7d5-09e99d974e45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:33.456867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cf7cb-5434-4b53-a7d5-09e99d974e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-876cf7cb-5434-4b53-a7d5-09e99d974e45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:33.469297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:33.470509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:33.471551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:33.474182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 115/229] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:50:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:33.483358 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cf7cb-5434-4b53-a7d5-09e99d974e45 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:33.492884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:33.540264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744fbd5-383a-40cb-89eb-769b73b5d5b3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:33.606659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9975b-7635-47a8-be52-ac13fac111a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:33.607352 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9975b-7635-47a8-be52-ac13fac111a5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:33.607642 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9975b-7635-47a8-be52-ac13fac111a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:33.607924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9975b-7635-47a8-be52-ac13fac111a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:33.608839 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9975b-7635-47a8-be52-ac13fac111a5 None admin] RBAC: Adding query filter params (role.id=4498e695105347909981a684f613b6a2, scope.project.id=5589046ca8fb418da62e5cef63cedc9f, user.id=5055afe87d8d489b839cf2e35647edc6) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:33.609158 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9975b-7635-47a8-be52-ac13fac111a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:33.611819 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9975b-7635-47a8-be52-ac13fac111a5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:33.624395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 115/230] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:33 2022] GET /identity/v3/role_assignments?user.id=5055afe87d8d489b839cf2e35647edc6&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=4498e695105347909981a684f613b6a2 => generated 259 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:33.652825 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cf7cb-5434-4b53-a7d5-09e99d974e45 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:33.673180 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:33.694591 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b951b-3202-44b7-a3f5-1d2bc16d70f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:33.695300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b951b-3202-44b7-a3f5-1d2bc16d70f4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:33.695651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b951b-3202-44b7-a3f5-1d2bc16d70f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:33.695928 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b951b-3202-44b7-a3f5-1d2bc16d70f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:33.696912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b951b-3202-44b7-a3f5-1d2bc16d70f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:33.697233 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b951b-3202-44b7-a3f5-1d2bc16d70f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:33.698668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b951b-3202-44b7-a3f5-1d2bc16d70f4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:33.709534 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 116/231] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:50:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:33.717326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9975b-7635-47a8-be52-ac13fac111a5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:33.902076 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9975b-7635-47a8-be52-ac13fac111a5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:33.947547 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe257d-8057-495f-861d-f11042c326bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:33.948821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe257d-8057-495f-861d-f11042c326bc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:33.949300 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe257d-8057-495f-861d-f11042c326bc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:33.949759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe257d-8057-495f-861d-f11042c326bc None admin] PATH_INFO: `/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:33.951392 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe257d-8057-495f-861d-f11042c326bc None admin] RBAC: Authorizing `identity:get_user(user_id=07f5fc86e79c434b9bebe6bbcfdd14b5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:33.953086 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe257d-8057-495f-861d-f11042c326bc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:33.955488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 116/232] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:33 2022] GET /identity/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5 => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:33.963800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b951b-3202-44b7-a3f5-1d2bc16d70f4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:34.122500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b951b-3202-44b7-a3f5-1d2bc16d70f4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:34.185111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f92e3f-1b53-45d1-a90b-30e5d904a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:34.187422 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92e3f-1b53-45d1-a90b-30e5d904a43a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:34.187923 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92e3f-1b53-45d1-a90b-30e5d904a43a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:34.188455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92e3f-1b53-45d1-a90b-30e5d904a43a None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:34.190939 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92e3f-1b53-45d1-a90b-30e5d904a43a None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:34.193929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92e3f-1b53-45d1-a90b-30e5d904a43a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:34.197637 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 117/233] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:33 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:34.210174 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe257d-8057-495f-861d-f11042c326bc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:34.238586 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:34.456410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe257d-8057-495f-861d-f11042c326bc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:34.545297 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ac00d-6787-4849-b977-b50658900e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:34.546511 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ac00d-6787-4849-b977-b50658900e01 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:34.546883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ac00d-6787-4849-b977-b50658900e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:34.547240 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ac00d-6787-4849-b977-b50658900e01 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f/users/07f5fc86e79c434b9bebe6bbcfdd14b5/roles/f4c6e6a3b05a42abb56447797f7ade15` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:34.555230 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ac00d-6787-4849-b977-b50658900e01 None admin] RBAC: Authorizing `identity:create_grant(project_id=5589046ca8fb418da62e5cef63cedc9f, user_id=07f5fc86e79c434b9bebe6bbcfdd14b5, role_id=f4c6e6a3b05a42abb56447797f7ade15)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:34.556893 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ac00d-6787-4849-b977-b50658900e01 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:34.677369 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 117/234] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:50:34 2022] PUT /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f/users/07f5fc86e79c434b9bebe6bbcfdd14b5/roles/f4c6e6a3b05a42abb56447797f7ade15 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:50:35.656162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f7d04-d515-438a-905d-8ab2573b1d89 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:35.656799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f7d04-d515-438a-905d-8ab2573b1d89 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:35.656799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f7d04-d515-438a-905d-8ab2573b1d89 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:35.658203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 118/235] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:35.665920 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791c02c-6b41-4c3e-8182-378ef46ae77d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:35.666438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791c02c-6b41-4c3e-8182-378ef46ae77d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:35.666512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791c02c-6b41-4c3e-8182-378ef46ae77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:35.675786 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7bb0b-4519-4b0c-b899-35307b2d5674 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:35.676191 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7bb0b-4519-4b0c-b899-35307b2d5674 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:35.676407 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7bb0b-4519-4b0c-b899-35307b2d5674 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:35.677881 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 119/236] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:35.684788 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f24d-3023-44ed-856d-e5adec57d664 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:35.685355 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f24d-3023-44ed-856d-e5adec57d664 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:35.685880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f24d-3023-44ed-856d-e5adec57d664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:35.704341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9791c02c-6b41-4c3e-8182-378ef46ae77d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:35.706524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9791c02c-6b41-4c3e-8182-378ef46ae77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:35.731548 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6c28f24d-3023-44ed-856d-e5adec57d664 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:35.733539 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6c28f24d-3023-44ed-856d-e5adec57d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:35.808138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:35.839438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 120/237] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:35.873288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28f24d-3023-44ed-856d-e5adec57d664 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:35.887461 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:35.898532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 118/238] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:35.951536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791c02c-6b41-4c3e-8182-378ef46ae77d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:36.041901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28f24d-3023-44ed-856d-e5adec57d664 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:36.090484 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17bb78-6b69-49eb-898e-7ce6c6d96fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:36.091686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17bb78-6b69-49eb-898e-7ce6c6d96fed None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:36.092169 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17bb78-6b69-49eb-898e-7ce6c6d96fed None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:36.092678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17bb78-6b69-49eb-898e-7ce6c6d96fed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:36.109560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17bb78-6b69-49eb-898e-7ce6c6d96fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1c17bb78-6b69-49eb-898e-7ce6c6d96fed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:36.123419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:36.125230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:36.126859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 121/239] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:50:36.131309 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17bb78-6b69-49eb-898e-7ce6c6d96fed None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:36.207256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791c02c-6b41-4c3e-8182-378ef46ae77d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:36.230889 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:36.304511 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e373d0-180c-4a16-a5c7-e2cbf78478fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:36.305365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e373d0-180c-4a16-a5c7-e2cbf78478fd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:36.305659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e373d0-180c-4a16-a5c7-e2cbf78478fd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:36.305949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e373d0-180c-4a16-a5c7-e2cbf78478fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:36.307279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e373d0-180c-4a16-a5c7-e2cbf78478fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:36.308832 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e373d0-180c-4a16-a5c7-e2cbf78478fd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:36.310598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 119/240] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:36.317074 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e373d0-180c-4a16-a5c7-e2cbf78478fd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:36.395240 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17bb78-6b69-49eb-898e-7ce6c6d96fed None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:36.511659 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b967e1-a82f-4bc6-bb8c-d4f15769def9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:36.512505 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:36.513547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b967e1-a82f-4bc6-bb8c-d4f15769def9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:36.513743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b967e1-a82f-4bc6-bb8c-d4f15769def9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:36.514001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b967e1-a82f-4bc6-bb8c-d4f15769def9 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:36.515106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b967e1-a82f-4bc6-bb8c-d4f15769def9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:36.515281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b967e1-a82f-4bc6-bb8c-d4f15769def9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:36.521912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b967e1-a82f-4bc6-bb8c-d4f15769def9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:36.548369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 122/241] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:36.557339 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b967e1-a82f-4bc6-bb8c-d4f15769def9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:36.578917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e373d0-180c-4a16-a5c7-e2cbf78478fd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:36.642460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b893d-5e46-462d-bb2a-b0f3aae00868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:36.643232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b893d-5e46-462d-bb2a-b0f3aae00868 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:36.643474 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b893d-5e46-462d-bb2a-b0f3aae00868 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:36.643759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b893d-5e46-462d-bb2a-b0f3aae00868 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:36.644539 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b893d-5e46-462d-bb2a-b0f3aae00868 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:36.646773 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b893d-5e46-462d-bb2a-b0f3aae00868 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:36.764975 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:36.770532 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b967e1-a82f-4bc6-bb8c-d4f15769def9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:36.783946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 120/242] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:50:36 2022] POST /identity/v3/users => generated 293 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:50:36.861125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa8384-01c4-4f55-a04c-85215257342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:36.862311 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa8384-01c4-4f55-a04c-85215257342d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:36.862838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa8384-01c4-4f55-a04c-85215257342d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:36.863322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa8384-01c4-4f55-a04c-85215257342d None admin] PATH_INFO: `/v3/users/41c74938dc4941f28a7738487b97da41` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:36.893184 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa8384-01c4-4f55-a04c-85215257342d None admin] RBAC: Authorizing `identity:get_user(user_id=41c74938dc4941f28a7738487b97da41)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:36.895221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa8384-01c4-4f55-a04c-85215257342d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:36.899785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 123/243] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:36 2022] GET /identity/v3/users/41c74938dc4941f28a7738487b97da41 => generated 297 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:36.905221 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b893d-5e46-462d-bb2a-b0f3aae00868 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:37.002046 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b546d-2177-452f-ba52-0f47aaba0e76 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:37.002511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b546d-2177-452f-ba52-0f47aaba0e76 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:37.002835 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b546d-2177-452f-ba52-0f47aaba0e76 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:37.006354 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 124/244] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:37.012268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83713387-5e43-44b0-9900-6f64c7bca14a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:37.012785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83713387-5e43-44b0-9900-6f64c7bca14a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:37.013144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83713387-5e43-44b0-9900-6f64c7bca14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:37.034775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-83713387-5e43-44b0-9900-6f64c7bca14a None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:37.037600 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-83713387-5e43-44b0-9900-6f64c7bca14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:37.106438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b893d-5e46-462d-bb2a-b0f3aae00868 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:37.149356 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:37.152901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 125/245] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:50:37.154533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eeb6d-89c1-4bd0-9339-ef33ef1b4533 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:37.155418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eeb6d-89c1-4bd0-9339-ef33ef1b4533 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:37.155418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eeb6d-89c1-4bd0-9339-ef33ef1b4533 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:37.165022 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 126/246] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:37.169858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbd84c-2547-4619-8e51-163c76a281c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:37.170515 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664045a-7652-4ecf-9341-7a13ecf35cc3 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:37.170650 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbd84c-2547-4619-8e51-163c76a281c5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:37.170820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664045a-7652-4ecf-9341-7a13ecf35cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:37.170916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbd84c-2547-4619-8e51-163c76a281c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:37.171127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbd84c-2547-4619-8e51-163c76a281c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:37.171223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664045a-7652-4ecf-9341-7a13ecf35cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:37.203573 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e664045a-7652-4ecf-9341-7a13ecf35cc3 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:37.204885 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e664045a-7652-4ecf-9341-7a13ecf35cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:37.295174 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbd84c-2547-4619-8e51-163c76a281c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:37.297774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbd84c-2547-4619-8e51-163c76a281c5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:37.317234 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 127/247] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:37.319070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664045a-7652-4ecf-9341-7a13ecf35cc3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:37.356835 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:37.471834 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 121/248] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:37.473295 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbd84c-2547-4619-8e51-163c76a281c5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:37.508348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664045a-7652-4ecf-9341-7a13ecf35cc3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:37.529241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:37.610402 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695d0e-33bb-4fbc-95af-ff7ee4d75b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:37.611451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695d0e-33bb-4fbc-95af-ff7ee4d75b80 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:37.613540 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695d0e-33bb-4fbc-95af-ff7ee4d75b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:37.613903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695d0e-33bb-4fbc-95af-ff7ee4d75b80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:37.643796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d695d0e-33bb-4fbc-95af-ff7ee4d75b80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3d695d0e-33bb-4fbc-95af-ff7ee4d75b80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:37.654757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:37.655669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:37.657253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 128/249] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:50:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:37.657910 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695d0e-33bb-4fbc-95af-ff7ee4d75b80 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:37.753027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:37.762551 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbd84c-2547-4619-8e51-163c76a281c5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:37.778351 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695d0e-33bb-4fbc-95af-ff7ee4d75b80 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:37.817527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30914410-da2d-4e42-b217-b4edf5b1d874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:37.819277 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30914410-da2d-4e42-b217-b4edf5b1d874 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:37.819854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30914410-da2d-4e42-b217-b4edf5b1d874 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:37.820547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30914410-da2d-4e42-b217-b4edf5b1d874 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:37.822015 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30914410-da2d-4e42-b217-b4edf5b1d874 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:37.823864 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30914410-da2d-4e42-b217-b4edf5b1d874 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:37.826310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 129/250] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:37 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:37.827758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30914410-da2d-4e42-b217-b4edf5b1d874 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:37.833692 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42245c-b139-492b-8a07-eb13207ab338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:37.834522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42245c-b139-492b-8a07-eb13207ab338 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:37.834819 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42245c-b139-492b-8a07-eb13207ab338 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:37.835100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42245c-b139-492b-8a07-eb13207ab338 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:37.848085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42245c-b139-492b-8a07-eb13207ab338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ba42245c-b139-492b-8a07-eb13207ab338 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:37.857014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:37.857964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:37.858789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:37.861598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 122/251] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:37.862678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42245c-b139-492b-8a07-eb13207ab338 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:37.957528 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30914410-da2d-4e42-b217-b4edf5b1d874 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:37.963530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:38.004361 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42245c-b139-492b-8a07-eb13207ab338 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:38.004545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711eee0-7918-4a3e-bc02-a6a869b13609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:38.004545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711eee0-7918-4a3e-bc02-a6a869b13609 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:38.004545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711eee0-7918-4a3e-bc02-a6a869b13609 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:38.004545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711eee0-7918-4a3e-bc02-a6a869b13609 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:38.004880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711eee0-7918-4a3e-bc02-a6a869b13609 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:38.004880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711eee0-7918-4a3e-bc02-a6a869b13609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:38.004880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711eee0-7918-4a3e-bc02-a6a869b13609 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:38.013211 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 130/252] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:38.014622 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711eee0-7918-4a3e-bc02-a6a869b13609 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:38.026312 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:38.088797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b5c71-fd59-48b1-962e-909e1f1a2400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:38.089576 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b5c71-fd59-48b1-962e-909e1f1a2400 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:38.090029 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b5c71-fd59-48b1-962e-909e1f1a2400 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:38.090452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b5c71-fd59-48b1-962e-909e1f1a2400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:38.091471 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b5c71-fd59-48b1-962e-909e1f1a2400 None admin] RBAC: Adding query filter params (role.id=8e0ef0ea0d26497db73b62d43d4fda22, scope.project.id=5589046ca8fb418da62e5cef63cedc9f, user.id=41c74938dc4941f28a7738487b97da41) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:38.091945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b5c71-fd59-48b1-962e-909e1f1a2400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:38.093878 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b5c71-fd59-48b1-962e-909e1f1a2400 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:38.110638 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 123/253] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:37 2022] GET /identity/v3/role_assignments?user.id=41c74938dc4941f28a7738487b97da41&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=8e0ef0ea0d26497db73b62d43d4fda22 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:38.111927 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b5c71-fd59-48b1-962e-909e1f1a2400 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:38.141821 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711eee0-7918-4a3e-bc02-a6a869b13609 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:38.182331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1717d1-0ab8-4559-bca8-5c57d347a217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:38.183450 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1717d1-0ab8-4559-bca8-5c57d347a217 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:38.183945 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1717d1-0ab8-4559-bca8-5c57d347a217 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:38.184545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1717d1-0ab8-4559-bca8-5c57d347a217 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:38.185947 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1717d1-0ab8-4559-bca8-5c57d347a217 None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:38.186649 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1717d1-0ab8-4559-bca8-5c57d347a217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:38.189401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1717d1-0ab8-4559-bca8-5c57d347a217 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:38.203399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 131/254] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:38.206127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea031-5f8d-4585-8013-072a4a3cb493 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:38.206849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea031-5f8d-4585-8013-072a4a3cb493 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:38.207526 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea031-5f8d-4585-8013-072a4a3cb493 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:38.210923 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 132/255] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:38.217517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28b0f7-c4ba-4fbd-8e90-97bf618f5769 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:38.217785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28b0f7-c4ba-4fbd-8e90-97bf618f5769 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:38.218207 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28b0f7-c4ba-4fbd-8e90-97bf618f5769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:38.252604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9c28b0f7-c4ba-4fbd-8e90-97bf618f5769 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:38.254706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9c28b0f7-c4ba-4fbd-8e90-97bf618f5769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:38.377866 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b5c71-fd59-48b1-962e-909e1f1a2400 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:38.402323 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:38.416383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 133/256] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:38.418358 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28b0f7-c4ba-4fbd-8e90-97bf618f5769 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:38.466557 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d45292-5c8f-4f5a-8547-58ae00ff5688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:38.467897 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d45292-5c8f-4f5a-8547-58ae00ff5688 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:38.470545 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d45292-5c8f-4f5a-8547-58ae00ff5688 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:38.471109 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d45292-5c8f-4f5a-8547-58ae00ff5688 None admin] PATH_INFO: `/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:38.474779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d45292-5c8f-4f5a-8547-58ae00ff5688 None admin] RBAC: Authorizing `identity:get_user(user_id=07f5fc86e79c434b9bebe6bbcfdd14b5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:38.478140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d45292-5c8f-4f5a-8547-58ae00ff5688 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:38.485455 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 124/257] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:38 2022] GET /identity/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5 => generated 283 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:38.486765 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d45292-5c8f-4f5a-8547-58ae00ff5688 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:38.553672 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:38.701292 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28b0f7-c4ba-4fbd-8e90-97bf618f5769 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:38.752699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d45292-5c8f-4f5a-8547-58ae00ff5688 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:38.795394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13856ac-8dff-4488-9046-1e45946b8421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:38.796251 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13856ac-8dff-4488-9046-1e45946b8421 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:38.796550 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13856ac-8dff-4488-9046-1e45946b8421 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:38.796894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13856ac-8dff-4488-9046-1e45946b8421 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:38.810839 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13856ac-8dff-4488-9046-1e45946b8421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:38.818787 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3868d-e451-4fe1-b428-cda079384c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:38.819717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3868d-e451-4fe1-b428-cda079384c01 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:38.819939 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3868d-e451-4fe1-b428-cda079384c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:38.820172 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3868d-e451-4fe1-b428-cda079384c01 None admin] PATH_INFO: `/v3/users/5055afe87d8d489b839cf2e35647edc6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b13856ac-8dff-4488-9046-1e45946b8421 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:38.823470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:38.824390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:38.825181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:38.826700 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3868d-e451-4fe1-b428-cda079384c01 None admin] RBAC: Authorizing `identity:get_user(user_id=5055afe87d8d489b839cf2e35647edc6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:38.826784 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 125/258] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:50:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:38.826852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3868d-e451-4fe1-b428-cda079384c01 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:38.826918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13856ac-8dff-4488-9046-1e45946b8421 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:38.828337 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 134/259] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:38 2022] GET /identity/v3/users/5055afe87d8d489b839cf2e35647edc6 => generated 297 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:38.839438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3868d-e451-4fe1-b428-cda079384c01 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:38.926963 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:38.963011 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13856ac-8dff-4488-9046-1e45946b8421 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:39.008913 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:39.015219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe147ead-fca8-4d1d-88e3-e4096e07daa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:39.016289 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe147ead-fca8-4d1d-88e3-e4096e07daa7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:39.016865 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe147ead-fca8-4d1d-88e3-e4096e07daa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:39.017548 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe147ead-fca8-4d1d-88e3-e4096e07daa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:39.091664 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3868d-e451-4fe1-b428-cda079384c01 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:39.117465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe147ead-fca8-4d1d-88e3-e4096e07daa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:39.121208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe147ead-fca8-4d1d-88e3-e4096e07daa7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:39.181995 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928de6b6-3aeb-45b4-aa0b-2e27a4103214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:39.183080 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928de6b6-3aeb-45b4-aa0b-2e27a4103214 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:39.183443 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928de6b6-3aeb-45b4-aa0b-2e27a4103214 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:39.183821 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928de6b6-3aeb-45b4-aa0b-2e27a4103214 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:39.184972 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928de6b6-3aeb-45b4-aa0b-2e27a4103214 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:39.184972 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928de6b6-3aeb-45b4-aa0b-2e27a4103214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:39.186810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928de6b6-3aeb-45b4-aa0b-2e27a4103214 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:39.199140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 135/260] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:50:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:39.207289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928de6b6-3aeb-45b4-aa0b-2e27a4103214 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:39.252843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:39.298212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 126/261] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:39.299488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe147ead-fca8-4d1d-88e3-e4096e07daa7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:39.429112 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe147ead-fca8-4d1d-88e3-e4096e07daa7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:39.480102 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f94126-37a1-44cf-ba78-35fc6f4f3984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:39.482711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f94126-37a1-44cf-ba78-35fc6f4f3984 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:39.483469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f94126-37a1-44cf-ba78-35fc6f4f3984 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:39.484895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f94126-37a1-44cf-ba78-35fc6f4f3984 None admin] PATH_INFO: `/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:39.487452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f94126-37a1-44cf-ba78-35fc6f4f3984 None admin] RBAC: Authorizing `identity:get_user(user_id=07f5fc86e79c434b9bebe6bbcfdd14b5)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:39.490927 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f94126-37a1-44cf-ba78-35fc6f4f3984 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:39.494049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 127/262] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:39 2022] GET /identity/v3/users/07f5fc86e79c434b9bebe6bbcfdd14b5 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:39.498659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f94126-37a1-44cf-ba78-35fc6f4f3984 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:39.507255 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928de6b6-3aeb-45b4-aa0b-2e27a4103214 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:39.542580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:39.553195 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd5c80-bd90-4175-8b14-f8af7fa21ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:39.555253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd5c80-bd90-4175-8b14-f8af7fa21ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:39.555830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd5c80-bd90-4175-8b14-f8af7fa21ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:39.556367 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd5c80-bd90-4175-8b14-f8af7fa21ad5 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:39.557950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd5c80-bd90-4175-8b14-f8af7fa21ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:39.559708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd5c80-bd90-4175-8b14-f8af7fa21ad5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:39.562605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 136/263] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:39 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:39.563758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd5c80-bd90-4175-8b14-f8af7fa21ad5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:39.663714 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:39.738900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd5c80-bd90-4175-8b14-f8af7fa21ad5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:39.788984 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c5f7e-1bbf-46b2-94a1-87289f7825f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:39.790252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c5f7e-1bbf-46b2-94a1-87289f7825f6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:39.791858 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c5f7e-1bbf-46b2-94a1-87289f7825f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:39.792426 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c5f7e-1bbf-46b2-94a1-87289f7825f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:39.843574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f94126-37a1-44cf-ba78-35fc6f4f3984 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:39.888054 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9feb-1a99-4b6d-87ea-bd2793a6230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:39.889229 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9feb-1a99-4b6d-87ea-bd2793a6230a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:39.889519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9feb-1a99-4b6d-87ea-bd2793a6230a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:39.889795 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9feb-1a99-4b6d-87ea-bd2793a6230a None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:39.891382 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab9feb-1a99-4b6d-87ea-bd2793a6230a None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:39.892849 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab9feb-1a99-4b6d-87ea-bd2793a6230a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:39.893399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:39.894770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 128/264] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:39 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:39.895378 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9feb-1a99-4b6d-87ea-bd2793a6230a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:39.909674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c5f7e-1bbf-46b2-94a1-87289f7825f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:39.911973 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c5f7e-1bbf-46b2-94a1-87289f7825f6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:40.042096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9feb-1a99-4b6d-87ea-bd2793a6230a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:40.055862 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 137/265] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:40.061859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c84f0-148d-42ff-af21-e0f63cad89c0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.062165 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c84f0-148d-42ff-af21-e0f63cad89c0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.062423 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c84f0-148d-42ff-af21-e0f63cad89c0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.064669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 138/266] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:40.065392 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:40.065587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c26e2b-8fd1-4d67-99ca-04dbaef95741 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.065809 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c26e2b-8fd1-4d67-99ca-04dbaef95741 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.066065 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c26e2b-8fd1-4d67-99ca-04dbaef95741 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.068338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 139/267] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:40.068338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c26e2b-8fd1-4d67-99ca-04dbaef95741 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:40.100927 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dddf11-aa24-4401-80a2-0de411fe4c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:40.101831 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dddf11-aa24-4401-80a2-0de411fe4c94 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.102148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dddf11-aa24-4401-80a2-0de411fe4c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.102452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dddf11-aa24-4401-80a2-0de411fe4c94 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f/users/5055afe87d8d489b839cf2e35647edc6/roles/4498e695105347909981a684f613b6a2` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.105298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dddf11-aa24-4401-80a2-0de411fe4c94 None admin] RBAC: Authorizing `identity:create_grant(project_id=5589046ca8fb418da62e5cef63cedc9f, user_id=5055afe87d8d489b839cf2e35647edc6, role_id=4498e695105347909981a684f613b6a2)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:40.107150 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dddf11-aa24-4401-80a2-0de411fe4c94 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:40.184156 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 129/268] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:50:39 2022] PUT /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f/users/5055afe87d8d489b839cf2e35647edc6/roles/4498e695105347909981a684f613b6a2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:50:40.185565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dddf11-aa24-4401-80a2-0de411fe4c94 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:40.216921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:40.228971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c26e2b-8fd1-4d67-99ca-04dbaef95741 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:40.286251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e830b09-7854-45c7-8335-67dda3c7288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:40.287457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e830b09-7854-45c7-8335-67dda3c7288a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.287953 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e830b09-7854-45c7-8335-67dda3c7288a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.288493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e830b09-7854-45c7-8335-67dda3c7288a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.289589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e830b09-7854-45c7-8335-67dda3c7288a None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=5589046ca8fb418da62e5cef63cedc9f, user.id=07f5fc86e79c434b9bebe6bbcfdd14b5) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:40.290127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e830b09-7854-45c7-8335-67dda3c7288a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:40.293198 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e830b09-7854-45c7-8335-67dda3c7288a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:40.301101 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 140/269] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:40 2022] GET /identity/v3/role_assignments?user.id=07f5fc86e79c434b9bebe6bbcfdd14b5&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:40.303383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605c48b-aa0c-43b1-b546-ef19aa6f6dd5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.303880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605c48b-aa0c-43b1-b546-ef19aa6f6dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.304475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605c48b-aa0c-43b1-b546-ef19aa6f6dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.323918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f605c48b-aa0c-43b1-b546-ef19aa6f6dd5 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:40.325513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f605c48b-aa0c-43b1-b546-ef19aa6f6dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:40.327085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dddf11-aa24-4401-80a2-0de411fe4c94 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:40.391162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7b546-6a91-4e69-a833-48ac6e3ea51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:40.392506 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7b546-6a91-4e69-a833-48ac6e3ea51b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.394825 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7b546-6a91-4e69-a833-48ac6e3ea51b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.396776 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7b546-6a91-4e69-a833-48ac6e3ea51b None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.402339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7b546-6a91-4e69-a833-48ac6e3ea51b None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:40.404440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7b546-6a91-4e69-a833-48ac6e3ea51b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:40.409610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 130/270] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:40 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:40.411327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0567b-6188-43c3-98c2-0575d5cfdc1c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.415464 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0567b-6188-43c3-98c2-0575d5cfdc1c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.416309 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0567b-6188-43c3-98c2-0575d5cfdc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.449023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 141/271] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:40.450354 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605c48b-aa0c-43b1-b546-ef19aa6f6dd5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:40.451057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-dbf0567b-6188-43c3-98c2-0575d5cfdc1c None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:40.452883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-dbf0567b-6188-43c3-98c2-0575d5cfdc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:40.502376 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:40.558964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 131/272] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:40.559842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0567b-6188-43c3-98c2-0575d5cfdc1c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:40.606632 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:40.620458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605c48b-aa0c-43b1-b546-ef19aa6f6dd5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:40.674726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620268f-82b1-4331-abcb-20369aeb1040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:40.675555 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620268f-82b1-4331-abcb-20369aeb1040 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.675922 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620268f-82b1-4331-abcb-20369aeb1040 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.676308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620268f-82b1-4331-abcb-20369aeb1040 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.677245 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620268f-82b1-4331-abcb-20369aeb1040 None admin] RBAC: Adding query filter params (role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=5589046ca8fb418da62e5cef63cedc9f, user.id=07f5fc86e79c434b9bebe6bbcfdd14b5) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:40.677635 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620268f-82b1-4331-abcb-20369aeb1040 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:40.679191 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620268f-82b1-4331-abcb-20369aeb1040 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:40.695932 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 142/273] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:40 2022] GET /identity/v3/role_assignments?user.id=07f5fc86e79c434b9bebe6bbcfdd14b5&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:40.695932 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620268f-82b1-4331-abcb-20369aeb1040 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:40.730004 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0567b-6188-43c3-98c2-0575d5cfdc1c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:40.772439 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b007cc2-c4bb-48e2-a413-b57cc807e684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:40.773635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b007cc2-c4bb-48e2-a413-b57cc807e684 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.774199 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b007cc2-c4bb-48e2-a413-b57cc807e684 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.774715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b007cc2-c4bb-48e2-a413-b57cc807e684 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.788561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b007cc2-c4bb-48e2-a413-b57cc807e684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1b007cc2-c4bb-48e2-a413-b57cc807e684 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:40.797380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:40.798353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:40.799147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:40.801914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 132/274] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:40.809444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b007cc2-c4bb-48e2-a413-b57cc807e684 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:40.846444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:40.875146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620268f-82b1-4331-abcb-20369aeb1040 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:40.931673 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b007cc2-c4bb-48e2-a413-b57cc807e684 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:40.947074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaff40-9e63-40e6-b0e1-b62c02b8c067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:40.948347 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaff40-9e63-40e6-b0e1-b62c02b8c067 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.948786 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaff40-9e63-40e6-b0e1-b62c02b8c067 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.949318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaff40-9e63-40e6-b0e1-b62c02b8c067 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.957678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eaff40-9e63-40e6-b0e1-b62c02b8c067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:40.960043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eaff40-9e63-40e6-b0e1-b62c02b8c067 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:40.961867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 143/275] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:40.968970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaff40-9e63-40e6-b0e1-b62c02b8c067 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:40.977703 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a86b5-7143-4152-8f49-d5cd76517e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:40.978753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a86b5-7143-4152-8f49-d5cd76517e5a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:40.979249 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a86b5-7143-4152-8f49-d5cd76517e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:40.979762 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a86b5-7143-4152-8f49-d5cd76517e5a None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:40.980774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a86b5-7143-4152-8f49-d5cd76517e5a None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:40.981305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a86b5-7143-4152-8f49-d5cd76517e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:40.983093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a86b5-7143-4152-8f49-d5cd76517e5a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:40.994396 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 133/276] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:41.000254 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a86b5-7143-4152-8f49-d5cd76517e5a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:41.013860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:41.180210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaff40-9e63-40e6-b0e1-b62c02b8c067 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:41.193975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:41.210894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a86b5-7143-4152-8f49-d5cd76517e5a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:41.238524 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79bb7d-300b-4429-9728-5a28910392a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:41.239280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79bb7d-300b-4429-9728-5a28910392a7 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:41.240129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79bb7d-300b-4429-9728-5a28910392a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:41.240581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79bb7d-300b-4429-9728-5a28910392a7 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:41.241405 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79bb7d-300b-4429-9728-5a28910392a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:41.242876 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79bb7d-300b-4429-9728-5a28910392a7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:41.265399 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba15e0-d439-4050-8691-caf9f8c79d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:41.266279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba15e0-d439-4050-8691-caf9f8c79d1c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:41.266618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba15e0-d439-4050-8691-caf9f8c79d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:41.266882 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba15e0-d439-4050-8691-caf9f8c79d1c None admin] PATH_INFO: `/v3/users/8ebb323a3474474cb3a0e74cf468fb69` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:41.292733 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba15e0-d439-4050-8691-caf9f8c79d1c None admin] RBAC: Authorizing `identity:get_user(user_id=8ebb323a3474474cb3a0e74cf468fb69)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:41.294970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba15e0-d439-4050-8691-caf9f8c79d1c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:41.297611 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 134/277] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:40 2022] GET /identity/v3/users/8ebb323a3474474cb3a0e74cf468fb69 => generated 293 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:41.305270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba15e0-d439-4050-8691-caf9f8c79d1c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:41.362880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 144/278] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:50:40 2022] POST /identity/v3/users => generated 293 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:50:41.378947 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5298db-46e4-4724-a1f3-c208c856c4e0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:41.379296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5298db-46e4-4724-a1f3-c208c856c4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:41.379520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5298db-46e4-4724-a1f3-c208c856c4e0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:41.382168 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 145/279] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:41.387977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c6cc7-5766-45e5-acd3-d0d8f42ec8fe None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:41.388306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c6cc7-5766-45e5-acd3-d0d8f42ec8fe None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:41.388577 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c6cc7-5766-45e5-acd3-d0d8f42ec8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:41.408288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d69c6cc7-5766-45e5-acd3-d0d8f42ec8fe None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:41.409727 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d69c6cc7-5766-45e5-acd3-d0d8f42ec8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:41.443904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba15e0-d439-4050-8691-caf9f8c79d1c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:41.497193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbe96e-99a1-4e93-9c57-3a1e74bd31ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:41.498280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbe96e-99a1-4e93-9c57-3a1e74bd31ab None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:41.498769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbe96e-99a1-4e93-9c57-3a1e74bd31ab None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:41.499242 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbe96e-99a1-4e93-9c57-3a1e74bd31ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:41.500424 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbe96e-99a1-4e93-9c57-3a1e74bd31ab None admin] RBAC: Adding query filter params (role.id=4498e695105347909981a684f613b6a2, user.id=8ebb323a3474474cb3a0e74cf468fb69, scope.system=all) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:41.500977 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbe96e-99a1-4e93-9c57-3a1e74bd31ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:41.503804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbe96e-99a1-4e93-9c57-3a1e74bd31ab None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:41.523742 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 135/280] 10.222.0.88 () {56 vars in 1308 bytes} [Thu Jun 16 14:50:41 2022] GET /identity/v3/role_assignments?user.id=8ebb323a3474474cb3a0e74cf468fb69&scope.system=all&role.id=4498e695105347909981a684f613b6a2 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:41.538925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 146/281] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:50:41.567885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbe96e-99a1-4e93-9c57-3a1e74bd31ab None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:41.595219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:41.705700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbe96e-99a1-4e93-9c57-3a1e74bd31ab None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:41.748659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750f17d-6ab0-4c96-8401-eae87006afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:41.750041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750f17d-6ab0-4c96-8401-eae87006afbc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:41.750499 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750f17d-6ab0-4c96-8401-eae87006afbc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:41.750960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750f17d-6ab0-4c96-8401-eae87006afbc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:41.765804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750f17d-6ab0-4c96-8401-eae87006afbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-e750f17d-6ab0-4c96-8401-eae87006afbc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:41.774603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:41.775558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:41.776400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 136/282] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:41.782384 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c6cc7-5766-45e5-acd3-d0d8f42ec8fe None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:41.798953 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:41.943886 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c6cc7-5766-45e5-acd3-d0d8f42ec8fe None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:42.004447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35bbac-1885-4901-831f-a72b63066024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:42.006131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35bbac-1885-4901-831f-a72b63066024 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:42.006240 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35bbac-1885-4901-831f-a72b63066024 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:42.006472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35bbac-1885-4901-831f-a72b63066024 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:42.007405 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35bbac-1885-4901-831f-a72b63066024 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:42.007723 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35bbac-1885-4901-831f-a72b63066024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:42.009511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35bbac-1885-4901-831f-a72b63066024 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:42.026629 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 147/283] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:42.035878 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750f17d-6ab0-4c96-8401-eae87006afbc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:42.173953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:42.209005 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750f17d-6ab0-4c96-8401-eae87006afbc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:42.264718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02c757-8826-4e50-9574-6456a8f7e992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:42.265523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02c757-8826-4e50-9574-6456a8f7e992 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:42.265819 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02c757-8826-4e50-9574-6456a8f7e992 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:42.266159 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02c757-8826-4e50-9574-6456a8f7e992 None admin] PATH_INFO: `/v3/users/41c74938dc4941f28a7738487b97da41` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:42.267417 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02c757-8826-4e50-9574-6456a8f7e992 None admin] RBAC: Authorizing `identity:get_user(user_id=41c74938dc4941f28a7738487b97da41)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:42.268945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02c757-8826-4e50-9574-6456a8f7e992 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:42.270998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 137/284] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:42 2022] GET /identity/v3/users/41c74938dc4941f28a7738487b97da41 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:42.277686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35bbac-1885-4901-831f-a72b63066024 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:42.392898 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:42.436247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35bbac-1885-4901-831f-a72b63066024 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:42.485351 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6dc8b5-abe9-437e-bbe4-e3c8a5bae22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:42.486230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6dc8b5-abe9-437e-bbe4-e3c8a5bae22a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:42.486512 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6dc8b5-abe9-437e-bbe4-e3c8a5bae22a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:42.486805 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6dc8b5-abe9-437e-bbe4-e3c8a5bae22a None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:42.488025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6dc8b5-abe9-437e-bbe4-e3c8a5bae22a None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:42.490067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6dc8b5-abe9-437e-bbe4-e3c8a5bae22a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:42.492872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 148/285] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:42 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:42.502174 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02c757-8826-4e50-9574-6456a8f7e992 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:42.721677 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02c757-8826-4e50-9574-6456a8f7e992 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:42.797724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b448c43-43b6-4e3f-a0cf-d098352adf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:42.798636 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b448c43-43b6-4e3f-a0cf-d098352adf36 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:42.798922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b448c43-43b6-4e3f-a0cf-d098352adf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:42.799226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b448c43-43b6-4e3f-a0cf-d098352adf36 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f/users/41c74938dc4941f28a7738487b97da41/roles/8e0ef0ea0d26497db73b62d43d4fda22` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:42.802172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b448c43-43b6-4e3f-a0cf-d098352adf36 None admin] RBAC: Authorizing `identity:create_grant(project_id=5589046ca8fb418da62e5cef63cedc9f, user_id=41c74938dc4941f28a7738487b97da41, role_id=8e0ef0ea0d26497db73b62d43d4fda22)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:42.803805 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b448c43-43b6-4e3f-a0cf-d098352adf36 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:42.915890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 138/286] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:50:42 2022] PUT /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f/users/41c74938dc4941f28a7738487b97da41/roles/8e0ef0ea0d26497db73b62d43d4fda22 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:50:43.334288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a88509-0f31-4c85-8cf7-ca9f8ae32d93 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:43.335086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a88509-0f31-4c85-8cf7-ca9f8ae32d93 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:43.335086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a88509-0f31-4c85-8cf7-ca9f8ae32d93 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:43.336556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 149/287] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:43.343058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801f78f-c80e-4a3d-a7ee-f756d7e4f529 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:43.343761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801f78f-c80e-4a3d-a7ee-f756d7e4f529 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:43.343761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801f78f-c80e-4a3d-a7ee-f756d7e4f529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:43.350961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:43.370025 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f801f78f-c80e-4a3d-a7ee-f756d7e4f529 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:43.371219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f801f78f-c80e-4a3d-a7ee-f756d7e4f529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:43.478018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 139/288] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:43.520505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a88509-0f31-4c85-8cf7-ca9f8ae32d93 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:43.746340 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a88509-0f31-4c85-8cf7-ca9f8ae32d93 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:43.800947 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df3dd2-2e09-4c16-9def-5c412d2636c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:43.802675 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df3dd2-2e09-4c16-9def-5c412d2636c1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:43.803204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df3dd2-2e09-4c16-9def-5c412d2636c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:43.803692 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df3dd2-2e09-4c16-9def-5c412d2636c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:43.812504 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:43.817138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df3dd2-2e09-4c16-9def-5c412d2636c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-10df3dd2-2e09-4c16-9def-5c412d2636c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:43.824982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:43.829736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:43.830565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 150/289] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:43.837436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801f78f-c80e-4a3d-a7ee-f756d7e4f529 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:43.998994 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801f78f-c80e-4a3d-a7ee-f756d7e4f529 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:44.057551 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:44.062545 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53b28b-74cb-4050-8f4f-ccb503c4131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:44.063239 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53b28b-74cb-4050-8f4f-ccb503c4131e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:44.063528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53b28b-74cb-4050-8f4f-ccb503c4131e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:44.063811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53b28b-74cb-4050-8f4f-ccb503c4131e None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:44.064753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53b28b-74cb-4050-8f4f-ccb503c4131e None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:44.065081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53b28b-74cb-4050-8f4f-ccb503c4131e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:44.066506 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53b28b-74cb-4050-8f4f-ccb503c4131e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:44.079323 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 140/290] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:44.085208 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df3dd2-2e09-4c16-9def-5c412d2636c1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:44.246599 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df3dd2-2e09-4c16-9def-5c412d2636c1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:44.299094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7533c8a-8b33-4188-806e-a04e6f3c11f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:44.299992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7533c8a-8b33-4188-806e-a04e6f3c11f9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:44.300304 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7533c8a-8b33-4188-806e-a04e6f3c11f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:44.300589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7533c8a-8b33-4188-806e-a04e6f3c11f9 None admin] PATH_INFO: `/v3/users/5055afe87d8d489b839cf2e35647edc6` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:44.301896 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7533c8a-8b33-4188-806e-a04e6f3c11f9 None admin] RBAC: Authorizing `identity:get_user(user_id=5055afe87d8d489b839cf2e35647edc6)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:44.303581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7533c8a-8b33-4188-806e-a04e6f3c11f9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:44.305722 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 151/291] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:44 2022] GET /identity/v3/users/5055afe87d8d489b839cf2e35647edc6 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:44.311888 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53b28b-74cb-4050-8f4f-ccb503c4131e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:44.493333 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53b28b-74cb-4050-8f4f-ccb503c4131e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:44.582140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0075e6b-d71c-47bc-8de8-1b5ca65a56fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:44.583282 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075e6b-d71c-47bc-8de8-1b5ca65a56fe None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:44.583754 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075e6b-d71c-47bc-8de8-1b5ca65a56fe None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:44.584258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075e6b-d71c-47bc-8de8-1b5ca65a56fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:44.654162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3167066-a984-4242-8dd9-95c10e41d3d3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:44.654672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3167066-a984-4242-8dd9-95c10e41d3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:44.654781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3167066-a984-4242-8dd9-95c10e41d3d3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:44.656463 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 152/292] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:44.670891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c435af-d924-46e5-a450-b6b3b4a5570a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:44.671285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c435af-d924-46e5-a450-b6b3b4a5570a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:44.671458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c435af-d924-46e5-a450-b6b3b4a5570a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:44.705444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0075e6b-d71c-47bc-8de8-1b5ca65a56fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:44.705444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0075e6b-d71c-47bc-8de8-1b5ca65a56fe None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:44.705702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:44.728931 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-77c435af-d924-46e5-a450-b6b3b4a5570a None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:44.730199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-77c435af-d924-46e5-a450-b6b3b4a5570a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:44.864631 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:44.887643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 153/293] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:44.933420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 141/294] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:44.940262 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c435af-d924-46e5-a450-b6b3b4a5570a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:44.945816 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0075e6b-d71c-47bc-8de8-1b5ca65a56fe None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:45.064801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:45.072938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0075e6b-d71c-47bc-8de8-1b5ca65a56fe None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:45.112830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c435af-d924-46e5-a450-b6b3b4a5570a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:45.132387 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37822615-f63e-47ae-9dc5-2b79aa4e0662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:45.134399 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37822615-f63e-47ae-9dc5-2b79aa4e0662 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.135054 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37822615-f63e-47ae-9dc5-2b79aa4e0662 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.135719 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37822615-f63e-47ae-9dc5-2b79aa4e0662 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.157944 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37822615-f63e-47ae-9dc5-2b79aa4e0662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:45.158543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd740fb-6fba-49c5-aada-6c169d80f435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:45.159535 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd740fb-6fba-49c5-aada-6c169d80f435 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.159928 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd740fb-6fba-49c5-aada-6c169d80f435 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.160338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd740fb-6fba-49c5-aada-6c169d80f435 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.161796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd740fb-6fba-49c5-aada-6c169d80f435 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:45.163519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd740fb-6fba-49c5-aada-6c169d80f435 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:45.170631 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 154/295] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:44 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-37822615-f63e-47ae-9dc5-2b79aa4e0662 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:45.170804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:45.172750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:45.173952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:45.174705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebc428-aa45-40be-a95d-a21068457296 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.174705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebc428-aa45-40be-a95d-a21068457296 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.174705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebc428-aa45-40be-a95d-a21068457296 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.174705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 155/296] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:45.174705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa38810-68ef-4283-8148-3573d72c900b None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.174705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa38810-68ef-4283-8148-3573d72c900b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.175223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa38810-68ef-4283-8148-3573d72c900b None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.175310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 142/297] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:45.175310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37822615-f63e-47ae-9dc5-2b79aa4e0662 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:45.175556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 156/298] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:45.176531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa38810-68ef-4283-8148-3573d72c900b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:45.284353 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:45.312334 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37822615-f63e-47ae-9dc5-2b79aa4e0662 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:45.348288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa38810-68ef-4283-8148-3573d72c900b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:45.351887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691a569-f071-4b04-907c-c18c908e40fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:45.352651 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691a569-f071-4b04-907c-c18c908e40fb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.352955 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691a569-f071-4b04-907c-c18c908e40fb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.353327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691a569-f071-4b04-907c-c18c908e40fb None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.354208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691a569-f071-4b04-907c-c18c908e40fb None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:45.354525 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691a569-f071-4b04-907c-c18c908e40fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:45.356542 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691a569-f071-4b04-907c-c18c908e40fb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:45.372802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 143/299] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:45.372802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c929cf-177f-4e06-a6c7-975e2349d7d6 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.372802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c929cf-177f-4e06-a6c7-975e2349d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.372802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c929cf-177f-4e06-a6c7-975e2349d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.399606 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-72c929cf-177f-4e06-a6c7-975e2349d7d6 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:45.401028 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-72c929cf-177f-4e06-a6c7-975e2349d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:45.414056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:45.414709 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59ff90-9537-4fe2-9254-5be3416d0a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:45.415619 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59ff90-9537-4fe2-9254-5be3416d0a4f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.416101 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59ff90-9537-4fe2-9254-5be3416d0a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.416625 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59ff90-9537-4fe2-9254-5be3416d0a4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.418099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59ff90-9537-4fe2-9254-5be3416d0a4f None admin] RBAC: Adding query filter params (role.id=4498e695105347909981a684f613b6a2, scope.project.id=5589046ca8fb418da62e5cef63cedc9f, user.id=5055afe87d8d489b839cf2e35647edc6) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:45.418421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59ff90-9537-4fe2-9254-5be3416d0a4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:45.420599 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59ff90-9537-4fe2-9254-5be3416d0a4f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:45.428751 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 157/300] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:45 2022] GET /identity/v3/role_assignments?user.id=5055afe87d8d489b839cf2e35647edc6&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=4498e695105347909981a684f613b6a2 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:45.430235 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80adc8e-f4bc-4b96-aaf2-63c8b8272a3f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.430517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80adc8e-f4bc-4b96-aaf2-63c8b8272a3f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.430879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80adc8e-f4bc-4b96-aaf2-63c8b8272a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.469306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a80adc8e-f4bc-4b96-aaf2-63c8b8272a3f None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:45.470651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a80adc8e-f4bc-4b96-aaf2-63c8b8272a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:45.537301 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 144/301] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:45.539881 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c929cf-177f-4e06-a6c7-975e2349d7d6 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:45.599696 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 158/302] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:45.600665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80adc8e-f4bc-4b96-aaf2-63c8b8272a3f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:45.718859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:45.726876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c929cf-177f-4e06-a6c7-975e2349d7d6 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:45.787490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd0bcc-b743-4eed-be4e-bbf1281a0faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:45.788440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd0bcc-b743-4eed-be4e-bbf1281a0faa None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.788727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd0bcc-b743-4eed-be4e-bbf1281a0faa None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.789006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd0bcc-b743-4eed-be4e-bbf1281a0faa None admin] PATH_INFO: `/v3/users/8ebb323a3474474cb3a0e74cf468fb69` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.790233 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd0bcc-b743-4eed-be4e-bbf1281a0faa None admin] RBAC: Authorizing `identity:get_user(user_id=8ebb323a3474474cb3a0e74cf468fb69)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:45.791687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd0bcc-b743-4eed-be4e-bbf1281a0faa None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:45.793370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 145/303] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:45 2022] GET /identity/v3/users/8ebb323a3474474cb3a0e74cf468fb69 => generated 293 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:45.800257 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd0bcc-b743-4eed-be4e-bbf1281a0faa None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:45.800416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80adc8e-f4bc-4b96-aaf2-63c8b8272a3f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:45.821160 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:45.846057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfa3f4-376c-4604-b968-8f8a17a58d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:45.847349 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfa3f4-376c-4604-b968-8f8a17a58d6d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.847858 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfa3f4-376c-4604-b968-8f8a17a58d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.848398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfa3f4-376c-4604-b968-8f8a17a58d6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.861090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfa3f4-376c-4604-b968-8f8a17a58d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fecfa3f4-376c-4604-b968-8f8a17a58d6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:45.870432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:45.871656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:45.872662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 159/304] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:45.873377 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfa3f4-376c-4604-b968-8f8a17a58d6d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:45.934131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd0bcc-b743-4eed-be4e-bbf1281a0faa None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:45.976804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e74e54-a1be-4a38-87bb-01b2f87e66d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:45.977489 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e74e54-a1be-4a38-87bb-01b2f87e66d3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:45.977775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e74e54-a1be-4a38-87bb-01b2f87e66d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:45.978054 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e74e54-a1be-4a38-87bb-01b2f87e66d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:45.979226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e74e54-a1be-4a38-87bb-01b2f87e66d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:45.980704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e74e54-a1be-4a38-87bb-01b2f87e66d3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:45.982526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 146/305] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:45.983221 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e74e54-a1be-4a38-87bb-01b2f87e66d3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:46.004929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfa3f4-376c-4604-b968-8f8a17a58d6d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:46.024561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:46.042772 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e857a9-1739-4ea0-b2e8-f1fcccff7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:46.043669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e857a9-1739-4ea0-b2e8-f1fcccff7455 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:46.043949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e857a9-1739-4ea0-b2e8-f1fcccff7455 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:46.044248 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e857a9-1739-4ea0-b2e8-f1fcccff7455 None admin] PATH_INFO: `/v3/system/users/8ebb323a3474474cb3a0e74cf468fb69/roles/4498e695105347909981a684f613b6a2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:46.045830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e857a9-1739-4ea0-b2e8-f1fcccff7455 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8ebb323a3474474cb3a0e74cf468fb69, role_id=4498e695105347909981a684f613b6a2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:46.047279 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e857a9-1739-4ea0-b2e8-f1fcccff7455 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:46.076961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:46.095544 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e74e54-a1be-4a38-87bb-01b2f87e66d3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:46.133205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613b315-ed1e-40c1-bf87-c9ed2e4131ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:46.134131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613b315-ed1e-40c1-bf87-c9ed2e4131ad None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:46.134415 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613b315-ed1e-40c1-bf87-c9ed2e4131ad None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:46.134691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613b315-ed1e-40c1-bf87-c9ed2e4131ad None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:46.135445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613b315-ed1e-40c1-bf87-c9ed2e4131ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:46.135753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613b315-ed1e-40c1-bf87-c9ed2e4131ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:46.137149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613b315-ed1e-40c1-bf87-c9ed2e4131ad None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:46.144465 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 160/306] 10.222.0.88 () {58 vars in 1334 bytes} [Thu Jun 16 14:50:45 2022] PUT /identity/v3/system/users/8ebb323a3474474cb3a0e74cf468fb69/roles/4498e695105347909981a684f613b6a2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:50:46.158589 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 147/307] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:46.158717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493224e7-5306-49f9-8f62-c5af35869914 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:46.158793 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613b315-ed1e-40c1-bf87-c9ed2e4131ad None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:46.158884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493224e7-5306-49f9-8f62-c5af35869914 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:46.158884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493224e7-5306-49f9-8f62-c5af35869914 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:46.158884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 161/308] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:46.158884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493224e7-5306-49f9-8f62-c5af35869914 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:46.297334 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613b315-ed1e-40c1-bf87-c9ed2e4131ad None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:46.315838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493224e7-5306-49f9-8f62-c5af35869914 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:46.343687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784dadd-4988-452c-9974-3edac73a9df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:46.344419 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784dadd-4988-452c-9974-3edac73a9df3 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:46.345701 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784dadd-4988-452c-9974-3edac73a9df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:46.345701 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784dadd-4988-452c-9974-3edac73a9df3 None admin] PATH_INFO: `/v3/groups` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:46.346112 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784dadd-4988-452c-9974-3edac73a9df3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:46.347597 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784dadd-4988-452c-9974-3edac73a9df3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:46.361708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddde3ca-bb3b-4678-80fe-f838a50bc4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:46.362688 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddde3ca-bb3b-4678-80fe-f838a50bc4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:46.363200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddde3ca-bb3b-4678-80fe-f838a50bc4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:46.367342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddde3ca-bb3b-4678-80fe-f838a50bc4f8 None admin] PATH_INFO: `/v3/users/b8f9b05586cb4d13999bfdce0c729b2b` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:46.378898 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddde3ca-bb3b-4678-80fe-f838a50bc4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=b8f9b05586cb4d13999bfdce0c729b2b)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:46.380938 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddde3ca-bb3b-4678-80fe-f838a50bc4f8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:46.382797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 162/309] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:46 2022] GET /identity/v3/users/b8f9b05586cb4d13999bfdce0c729b2b => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:46.384296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd107d-08c3-4b4c-b9a7-8ed758199b71 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:46.384581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd107d-08c3-4b4c-b9a7-8ed758199b71 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:46.384857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd107d-08c3-4b4c-b9a7-8ed758199b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:46.402473 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7ebd107d-08c3-4b4c-b9a7-8ed758199b71 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:46.403636 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7ebd107d-08c3-4b4c-b9a7-8ed758199b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:46.420873 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 148/310] 10.222.0.88 () {60 vars in 1134 bytes} [Thu Jun 16 14:50:46 2022] POST /identity/v3/groups => generated 229 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:50:46.420873 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784dadd-4988-452c-9974-3edac73a9df3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:46.431181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:46.458945 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:46.511533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 163/311] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:46.534783 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd107d-08c3-4b4c-b9a7-8ed758199b71 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:46.581654 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784dadd-4988-452c-9974-3edac73a9df3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:46.634773 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f0339-dca1-49ab-b5d9-3c5213aa6437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:46.636001 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f0339-dca1-49ab-b5d9-3c5213aa6437 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:46.636454 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f0339-dca1-49ab-b5d9-3c5213aa6437 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:46.636755 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f0339-dca1-49ab-b5d9-3c5213aa6437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:46.637654 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f0339-dca1-49ab-b5d9-3c5213aa6437 None admin] RBAC: Adding query filter params (role.id=8e0ef0ea0d26497db73b62d43d4fda22, user.id=b8f9b05586cb4d13999bfdce0c729b2b, scope.system=all) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:46.638249 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f0339-dca1-49ab-b5d9-3c5213aa6437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:46.640541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f0339-dca1-49ab-b5d9-3c5213aa6437 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:46.652448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 149/312] 10.222.0.88 () {56 vars in 1308 bytes} [Thu Jun 16 14:50:46 2022] GET /identity/v3/role_assignments?user.id=b8f9b05586cb4d13999bfdce0c729b2b&scope.system=all&role.id=8e0ef0ea0d26497db73b62d43d4fda22 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:46.666093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd107d-08c3-4b4c-b9a7-8ed758199b71 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:46.712083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c394db7-a1a1-4a53-b5f2-b87074a6f693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:46.713634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c394db7-a1a1-4a53-b5f2-b87074a6f693 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:46.714181 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c394db7-a1a1-4a53-b5f2-b87074a6f693 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:46.714693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c394db7-a1a1-4a53-b5f2-b87074a6f693 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:46.728660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c394db7-a1a1-4a53-b5f2-b87074a6f693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-0c394db7-a1a1-4a53-b5f2-b87074a6f693 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:46.737174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:46.738146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:46.741342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:46.747129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 164/313] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:50:46.748510 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f0339-dca1-49ab-b5d9-3c5213aa6437 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:46.831499 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:46.870366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f0339-dca1-49ab-b5d9-3c5213aa6437 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:46.912018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cf361-6bab-4dbb-847b-914025a99b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:46.912480 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cf361-6bab-4dbb-847b-914025a99b32 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:46.912775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cf361-6bab-4dbb-847b-914025a99b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:46.913062 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cf361-6bab-4dbb-847b-914025a99b32 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:46.913835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0cf361-6bab-4dbb-847b-914025a99b32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:46.914162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0cf361-6bab-4dbb-847b-914025a99b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:46.915566 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0cf361-6bab-4dbb-847b-914025a99b32 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:46.925372 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 150/314] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:46.930993 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c394db7-a1a1-4a53-b5f2-b87074a6f693 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:46.965581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:47.043799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c394db7-a1a1-4a53-b5f2-b87074a6f693 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:47.081704 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f05e3-90ce-4164-a876-3202223bd2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:47.082591 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f05e3-90ce-4164-a876-3202223bd2ba None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:47.082876 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f05e3-90ce-4164-a876-3202223bd2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:47.083674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f05e3-90ce-4164-a876-3202223bd2ba None admin] PATH_INFO: `/v3/users/41c74938dc4941f28a7738487b97da41` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:47.084983 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f05e3-90ce-4164-a876-3202223bd2ba None admin] RBAC: Authorizing `identity:get_user(user_id=41c74938dc4941f28a7738487b97da41)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:47.086467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f05e3-90ce-4164-a876-3202223bd2ba None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:47.088437 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 165/315] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:46 2022] GET /identity/v3/users/41c74938dc4941f28a7738487b97da41 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:47.093869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cf361-6bab-4dbb-847b-914025a99b32 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:47.203336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cf361-6bab-4dbb-847b-914025a99b32 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:47.234673 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:47.241833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f4885-4a0f-494c-a86b-ac7a1efc31a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:47.242501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f4885-4a0f-494c-a86b-ac7a1efc31a6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:47.242793 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f4885-4a0f-494c-a86b-ac7a1efc31a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:47.243078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f4885-4a0f-494c-a86b-ac7a1efc31a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:47.316684 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f4885-4a0f-494c-a86b-ac7a1efc31a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:47.319131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f4885-4a0f-494c-a86b-ac7a1efc31a6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:47.433545 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 151/316] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:47.434914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f05e3-90ce-4164-a876-3202223bd2ba None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:47.550687 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:47.554819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f05e3-90ce-4164-a876-3202223bd2ba None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:47.590325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7c8f6-7727-41a0-9cb5-4eec77a7684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:47.591199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7c8f6-7727-41a0-9cb5-4eec77a7684b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:47.591490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7c8f6-7727-41a0-9cb5-4eec77a7684b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:47.591792 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7c8f6-7727-41a0-9cb5-4eec77a7684b None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:47.593005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7c8f6-7727-41a0-9cb5-4eec77a7684b None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:47.594461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7c8f6-7727-41a0-9cb5-4eec77a7684b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:47.600341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 166/317] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:47 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:47.603432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f4885-4a0f-494c-a86b-ac7a1efc31a6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:47.634886 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:47.711477 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f4885-4a0f-494c-a86b-ac7a1efc31a6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:47.749392 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614452f-3ba8-4828-98a1-f30678ef0832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:47.750064 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614452f-3ba8-4828-98a1-f30678ef0832 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:47.750357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614452f-3ba8-4828-98a1-f30678ef0832 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:47.750643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614452f-3ba8-4828-98a1-f30678ef0832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:47.751492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614452f-3ba8-4828-98a1-f30678ef0832 None admin] RBAC: Adding query filter params (role.id=8e0ef0ea0d26497db73b62d43d4fda22, scope.project.id=5589046ca8fb418da62e5cef63cedc9f, user.id=41c74938dc4941f28a7738487b97da41) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:47.751838 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614452f-3ba8-4828-98a1-f30678ef0832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:47.753539 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614452f-3ba8-4828-98a1-f30678ef0832 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:47.762389 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 152/318] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:50:47 2022] GET /identity/v3/role_assignments?user.id=41c74938dc4941f28a7738487b97da41&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=8e0ef0ea0d26497db73b62d43d4fda22 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:48.438947 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e98227-1416-42b9-acc9-161164aa4f3c None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:48.439550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e98227-1416-42b9-acc9-161164aa4f3c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:48.439874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e98227-1416-42b9-acc9-161164aa4f3c None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:48.443265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 167/319] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:48.449109 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6690b-bb06-4e2b-936a-64bd66a1ba8c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:48.449329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6690b-bb06-4e2b-936a-64bd66a1ba8c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:48.449612 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6690b-bb06-4e2b-936a-64bd66a1ba8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:48.471828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a5b6690b-bb06-4e2b-936a-64bd66a1ba8c None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:48.473430 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a5b6690b-bb06-4e2b-936a-64bd66a1ba8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:48.592490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 153/320] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:48.619456 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e98227-1416-42b9-acc9-161164aa4f3c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:48.772469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e98227-1416-42b9-acc9-161164aa4f3c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:48.831971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41186a-3868-4bde-b385-d30f22298e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:48.836387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41186a-3868-4bde-b385-d30f22298e5d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:48.836909 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41186a-3868-4bde-b385-d30f22298e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:48.837090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41186a-3868-4bde-b385-d30f22298e5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:48.856343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41186a-3868-4bde-b385-d30f22298e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-bb41186a-3868-4bde-b385-d30f22298e5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:48.870661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:48.872620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:48.874155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 168/321] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:50:48.882248 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6690b-bb06-4e2b-936a-64bd66a1ba8c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:48.917943 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:49.014943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5675f-8ae0-4062-a27c-abac837650c3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.015254 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5675f-8ae0-4062-a27c-abac837650c3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.015539 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5675f-8ae0-4062-a27c-abac837650c3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.017047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 169/322] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:49.023474 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07377548-a6a7-4423-9afa-f61248f61aa1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.023726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07377548-a6a7-4423-9afa-f61248f61aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.023939 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07377548-a6a7-4423-9afa-f61248f61aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.032790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6690b-bb06-4e2b-936a-64bd66a1ba8c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:49.045184 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-07377548-a6a7-4423-9afa-f61248f61aa1 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:49.046445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-07377548-a6a7-4423-9afa-f61248f61aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:49.075690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:49.079042 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7faaa1-a6cf-4645-8189-d15cb5d369d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:49.079921 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7faaa1-a6cf-4645-8189-d15cb5d369d2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.080417 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7faaa1-a6cf-4645-8189-d15cb5d369d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.080896 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7faaa1-a6cf-4645-8189-d15cb5d369d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.082035 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7faaa1-a6cf-4645-8189-d15cb5d369d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:49.082476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7faaa1-a6cf-4645-8189-d15cb5d369d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:49.084561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7faaa1-a6cf-4645-8189-d15cb5d369d2 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:49.094696 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 154/323] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:49.096170 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dc698-97f4-49a9-abd5-4c685020243e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.096469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dc698-97f4-49a9-abd5-4c685020243e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.096752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dc698-97f4-49a9-abd5-4c685020243e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.098018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 155/324] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:49.101913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dc698-97f4-49a9-abd5-4c685020243e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:49.148149 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 170/325] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:49.149553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa9cde-c5f1-4e98-82b0-ca7c20b6bc86 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.149932 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa9cde-c5f1-4e98-82b0-ca7c20b6bc86 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.150169 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa9cde-c5f1-4e98-82b0-ca7c20b6bc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.167828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1baa9cde-c5f1-4e98-82b0-ca7c20b6bc86 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:49.169061 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1baa9cde-c5f1-4e98-82b0-ca7c20b6bc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:49.222630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:49.246911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dc698-97f4-49a9-abd5-4c685020243e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:49.267710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 171/326] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:49.268989 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa9cde-c5f1-4e98-82b0-ca7c20b6bc86 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:49.293145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c73f6-34b3-4914-bf69-e2e83d8ea6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:49.294706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c73f6-34b3-4914-bf69-e2e83d8ea6cb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.295206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c73f6-34b3-4914-bf69-e2e83d8ea6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.295718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c73f6-34b3-4914-bf69-e2e83d8ea6cb None admin] PATH_INFO: `/v3/users/8ebb323a3474474cb3a0e74cf468fb69` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.297891 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c73f6-34b3-4914-bf69-e2e83d8ea6cb None admin] RBAC: Authorizing `identity:get_user(user_id=8ebb323a3474474cb3a0e74cf468fb69)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:49.300604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c73f6-34b3-4914-bf69-e2e83d8ea6cb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:49.308304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 156/327] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:49 2022] GET /identity/v3/users/8ebb323a3474474cb3a0e74cf468fb69 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:49.308304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c73f6-34b3-4914-bf69-e2e83d8ea6cb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:49.375443 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:49.394443 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa9cde-c5f1-4e98-82b0-ca7c20b6bc86 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:49.438710 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c73f6-34b3-4914-bf69-e2e83d8ea6cb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:49.446266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4816487-474c-441c-b0f7-23f6d3ec2577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:49.447095 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4816487-474c-441c-b0f7-23f6d3ec2577 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.447376 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4816487-474c-441c-b0f7-23f6d3ec2577 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.447655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4816487-474c-441c-b0f7-23f6d3ec2577 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.448869 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4816487-474c-441c-b0f7-23f6d3ec2577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:49.450360 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4816487-474c-441c-b0f7-23f6d3ec2577 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:49.452048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 172/328] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:50:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:49.452864 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4816487-474c-441c-b0f7-23f6d3ec2577 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:49.485436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d88e7-fc19-42d7-af68-eaf58f31e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:49.486638 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d88e7-fc19-42d7-af68-eaf58f31e0aa None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.487086 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d88e7-fc19-42d7-af68-eaf58f31e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.487513 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d88e7-fc19-42d7-af68-eaf58f31e0aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.503882 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d88e7-fc19-42d7-af68-eaf58f31e0aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1b9d88e7-fc19-42d7-af68-eaf58f31e0aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:49.518092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:49.519172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:49.519991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:49.521839 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 157/329] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:49.522126 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d88e7-fc19-42d7-af68-eaf58f31e0aa None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:49.555466 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:49.585381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4816487-474c-441c-b0f7-23f6d3ec2577 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:49.659844 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8199b4-3ac7-4620-aca0-89f29ca97761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:49.660714 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8199b4-3ac7-4620-aca0-89f29ca97761 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.661761 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8199b4-3ac7-4620-aca0-89f29ca97761 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.662220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8199b4-3ac7-4620-aca0-89f29ca97761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.663650 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8199b4-3ac7-4620-aca0-89f29ca97761 None admin] RBAC: Adding query filter params (role.id=4498e695105347909981a684f613b6a2, user.id=8ebb323a3474474cb3a0e74cf468fb69, scope.system=all) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:49.664160 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8199b4-3ac7-4620-aca0-89f29ca97761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:49.666746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8199b4-3ac7-4620-aca0-89f29ca97761 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:49.686944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 173/330] 10.222.0.88 () {56 vars in 1308 bytes} [Thu Jun 16 14:50:49 2022] GET /identity/v3/role_assignments?user.id=8ebb323a3474474cb3a0e74cf468fb69&scope.system=all&role.id=4498e695105347909981a684f613b6a2 => generated 510 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:49.687942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8199b4-3ac7-4620-aca0-89f29ca97761 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:49.706586 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:49.730417 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d88e7-fc19-42d7-af68-eaf58f31e0aa None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:49.780248 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ea179-16e4-48b5-8eee-8e4be9b6cece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:49.781020 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ea179-16e4-48b5-8eee-8e4be9b6cece None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.781318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ea179-16e4-48b5-8eee-8e4be9b6cece None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.781683 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ea179-16e4-48b5-8eee-8e4be9b6cece None admin] PATH_INFO: `/v3/groups` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.782405 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ea179-16e4-48b5-8eee-8e4be9b6cece None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:49.783918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ea179-16e4-48b5-8eee-8e4be9b6cece None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:49.857621 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8199b4-3ac7-4620-aca0-89f29ca97761 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:49.894386 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 158/331] 10.222.0.88 () {60 vars in 1134 bytes} [Thu Jun 16 14:50:49 2022] POST /identity/v3/groups => generated 226 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:50:49.903720 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29153b71-acbe-4020-9538-8878b92dfc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:49.904809 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153b71-acbe-4020-9538-8878b92dfc9b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:49.905252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153b71-acbe-4020-9538-8878b92dfc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:49.905707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153b71-acbe-4020-9538-8878b92dfc9b None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:49.906640 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29153b71-acbe-4020-9538-8878b92dfc9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:49.907153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29153b71-acbe-4020-9538-8878b92dfc9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:49.908837 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29153b71-acbe-4020-9538-8878b92dfc9b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:49.919730 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 174/332] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:49.926985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ea179-16e4-48b5-8eee-8e4be9b6cece None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:50.021662 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:50.049085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ea179-16e4-48b5-8eee-8e4be9b6cece None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:50.093045 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034b34c-cc5c-404a-8a20-a52a12b7e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:50.093947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034b34c-cc5c-404a-8a20-a52a12b7e60d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:50.094242 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034b34c-cc5c-404a-8a20-a52a12b7e60d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:50.094527 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034b34c-cc5c-404a-8a20-a52a12b7e60d None admin] PATH_INFO: `/v3/users/b8f9b05586cb4d13999bfdce0c729b2b` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:50.095818 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034b34c-cc5c-404a-8a20-a52a12b7e60d None admin] RBAC: Authorizing `identity:get_user(user_id=b8f9b05586cb4d13999bfdce0c729b2b)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:50.097519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034b34c-cc5c-404a-8a20-a52a12b7e60d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:50.100445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 159/333] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:49 2022] GET /identity/v3/users/b8f9b05586cb4d13999bfdce0c729b2b => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:50.106071 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29153b71-acbe-4020-9538-8878b92dfc9b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:50.197626 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:50.227817 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29153b71-acbe-4020-9538-8878b92dfc9b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:50.278307 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349428c-fb0d-4a8d-8be9-5848ebc2f861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:50.279196 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349428c-fb0d-4a8d-8be9-5848ebc2f861 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:50.279483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349428c-fb0d-4a8d-8be9-5848ebc2f861 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:50.279773 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349428c-fb0d-4a8d-8be9-5848ebc2f861 None admin] PATH_INFO: `/v3/system/users/b8f9b05586cb4d13999bfdce0c729b2b/roles/8e0ef0ea0d26497db73b62d43d4fda22` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:50.281345 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349428c-fb0d-4a8d-8be9-5848ebc2f861 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b8f9b05586cb4d13999bfdce0c729b2b, role_id=8e0ef0ea0d26497db73b62d43d4fda22)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:50.282759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349428c-fb0d-4a8d-8be9-5848ebc2f861 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:50.418287 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 175/334] 10.222.0.88 () {58 vars in 1334 bytes} [Thu Jun 16 14:50:50 2022] PUT /identity/v3/system/users/b8f9b05586cb4d13999bfdce0c729b2b/roles/8e0ef0ea0d26497db73b62d43d4fda22 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:50:53.410043 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425d744-3e7d-42dc-a8d1-c028d56dec8a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:53.411592 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425d744-3e7d-42dc-a8d1-c028d56dec8a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:53.412565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425d744-3e7d-42dc-a8d1-c028d56dec8a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:53.415284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 160/335] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:53.421926 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0637021-7aa8-4ab0-908b-8e94f50def7f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:53.423297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0637021-7aa8-4ab0-908b-8e94f50def7f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:53.424032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0637021-7aa8-4ab0-908b-8e94f50def7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:53.466019 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a0637021-7aa8-4ab0-908b-8e94f50def7f None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:53.468458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a0637021-7aa8-4ab0-908b-8e94f50def7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:53.574572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba819bd4-521d-48c2-8088-c6fd09a88b55 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:53.575205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba819bd4-521d-48c2-8088-c6fd09a88b55 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:53.575724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba819bd4-521d-48c2-8088-c6fd09a88b55 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:53.577718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 161/336] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:53.584704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f580fc-8805-45a9-b5de-091ed74ac96a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:53.585435 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f580fc-8805-45a9-b5de-091ed74ac96a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:53.585923 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f580fc-8805-45a9-b5de-091ed74ac96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:53.609692 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 176/337] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:53.616394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-83f580fc-8805-45a9-b5de-091ed74ac96a None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:53.617957 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-83f580fc-8805-45a9-b5de-091ed74ac96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:53.676538 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0637021-7aa8-4ab0-908b-8e94f50def7f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:53.734777 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 162/338] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:53.802782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f580fc-8805-45a9-b5de-091ed74ac96a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:53.813842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:53.889288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0637021-7aa8-4ab0-908b-8e94f50def7f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:53.951781 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:53.970540 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3ac83-1a25-435a-9f96-11826081dc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:53.971717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3ac83-1a25-435a-9f96-11826081dc6d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:53.971978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3ac83-1a25-435a-9f96-11826081dc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:53.972283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3ac83-1a25-435a-9f96-11826081dc6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:53.997203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3ac83-1a25-435a-9f96-11826081dc6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:54.005652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f580fc-8805-45a9-b5de-091ed74ac96a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-aee3ac83-1a25-435a-9f96-11826081dc6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:54.005927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:54.007404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:54.008715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:54.013276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 177/339] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:50:54.021032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3ac83-1a25-435a-9f96-11826081dc6d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:54.077505 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0c04a-b2ed-4d04-8126-523e18dd13cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:54.078688 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c04a-b2ed-4d04-8126-523e18dd13cd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:54.079184 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c04a-b2ed-4d04-8126-523e18dd13cd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:54.079665 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0c04a-b2ed-4d04-8126-523e18dd13cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:54.108842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0c04a-b2ed-4d04-8126-523e18dd13cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-bdc0c04a-b2ed-4d04-8126-523e18dd13cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:54.117804 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:54.118749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:54.119577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:54.126460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 163/340] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:50:54.135089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0c04a-b2ed-4d04-8126-523e18dd13cd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:54.257075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3ac83-1a25-435a-9f96-11826081dc6d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:54.319043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:54.337634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e3d0a-d6b4-4dd4-ae82-251d5727e867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:54.339254 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3d0a-d6b4-4dd4-ae82-251d5727e867 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:54.339780 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3d0a-d6b4-4dd4-ae82-251d5727e867 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:54.344670 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3d0a-d6b4-4dd4-ae82-251d5727e867 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:54.346521 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e3d0a-d6b4-4dd4-ae82-251d5727e867 None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:54.347218 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e3d0a-d6b4-4dd4-ae82-251d5727e867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:54.350279 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e3d0a-d6b4-4dd4-ae82-251d5727e867 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:54.369142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 178/341] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:54.378188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e3d0a-d6b4-4dd4-ae82-251d5727e867 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:54.401483 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0c04a-b2ed-4d04-8126-523e18dd13cd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:54.476677 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:54.485511 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a754dd-65c3-400f-9c03-73bc041b1d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:54.486901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a754dd-65c3-400f-9c03-73bc041b1d8c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:54.497715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a754dd-65c3-400f-9c03-73bc041b1d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:54.497715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a754dd-65c3-400f-9c03-73bc041b1d8c None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:54.497715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a754dd-65c3-400f-9c03-73bc041b1d8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:54.497715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a754dd-65c3-400f-9c03-73bc041b1d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:54.497715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a754dd-65c3-400f-9c03-73bc041b1d8c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:54.519104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 164/342] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:54.529976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a754dd-65c3-400f-9c03-73bc041b1d8c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:54.581684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e3d0a-d6b4-4dd4-ae82-251d5727e867 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:54.639202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a816bd3-6826-45bc-8497-8a16d45d8044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:54.640148 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a816bd3-6826-45bc-8497-8a16d45d8044 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:54.640604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a816bd3-6826-45bc-8497-8a16d45d8044 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:54.641033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a816bd3-6826-45bc-8497-8a16d45d8044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:54.708708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a754dd-65c3-400f-9c03-73bc041b1d8c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:54.761333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a816bd3-6826-45bc-8497-8a16d45d8044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:54.772339 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a816bd3-6826-45bc-8497-8a16d45d8044 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:54.772501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef544b4-e242-44c5-9f91-29536beeaabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:54.772501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef544b4-e242-44c5-9f91-29536beeaabc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:54.772501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef544b4-e242-44c5-9f91-29536beeaabc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:54.772501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef544b4-e242-44c5-9f91-29536beeaabc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:54.776713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:54.896271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:54.896271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef544b4-e242-44c5-9f91-29536beeaabc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:54.900248 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef544b4-e242-44c5-9f91-29536beeaabc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:54.947245 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 179/343] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:50:54.949103 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21f062-7adc-4e9e-ac85-165f54b09698 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:54.949597 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21f062-7adc-4e9e-ac85-165f54b09698 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:54.949873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21f062-7adc-4e9e-ac85-165f54b09698 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:54.952360 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 180/344] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:54.953766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce51685-a68e-4e7c-acd7-5a588a5a3305 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:54.954260 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce51685-a68e-4e7c-acd7-5a588a5a3305 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:54.954746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce51685-a68e-4e7c-acd7-5a588a5a3305 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:54.958359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 181/345] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:54.959632 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce51685-a68e-4e7c-acd7-5a588a5a3305 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:55.065967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 165/346] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:55.068030 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88912e7e-2a1b-4216-9904-f3d3c641735a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.068345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88912e7e-2a1b-4216-9904-f3d3c641735a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.068638 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88912e7e-2a1b-4216-9904-f3d3c641735a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.083987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:55.086702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-88912e7e-2a1b-4216-9904-f3d3c641735a None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:55.087925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-88912e7e-2a1b-4216-9904-f3d3c641735a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:55.097783 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce51685-a68e-4e7c-acd7-5a588a5a3305 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:55.141499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c991e5-79c5-44d2-ad6f-396d19c4b392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:55.142758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c991e5-79c5-44d2-ad6f-396d19c4b392 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.143245 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c991e5-79c5-44d2-ad6f-396d19c4b392 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.143724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c991e5-79c5-44d2-ad6f-396d19c4b392 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.146030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c991e5-79c5-44d2-ad6f-396d19c4b392 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:55.147972 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c991e5-79c5-44d2-ad6f-396d19c4b392 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:55.156261 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 182/347] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:54 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:55.156261 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd412f-ba19-46a9-a882-bb43c46d8daf None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.156261 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd412f-ba19-46a9-a882-bb43c46d8daf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.156261 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd412f-ba19-46a9-a882-bb43c46d8daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.171212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 166/348] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:55.172599 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88912e7e-2a1b-4216-9904-f3d3c641735a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:55.179428 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-40fd412f-ba19-46a9-a882-bb43c46d8daf None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:55.180683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-40fd412f-ba19-46a9-a882-bb43c46d8daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:55.202011 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:55.268589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 183/349] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:55.269698 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd412f-ba19-46a9-a882-bb43c46d8daf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:55.304205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88912e7e-2a1b-4216-9904-f3d3c641735a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:55.344073 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78038e8-d70c-40ad-b9f6-2104ac1f4b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:55.345361 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78038e8-d70c-40ad-b9f6-2104ac1f4b3a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.345871 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78038e8-d70c-40ad-b9f6-2104ac1f4b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.346560 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78038e8-d70c-40ad-b9f6-2104ac1f4b3a None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.348014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78038e8-d70c-40ad-b9f6-2104ac1f4b3a None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:55.350833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78038e8-d70c-40ad-b9f6-2104ac1f4b3a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:55.352941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 167/350] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:55 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:55.354027 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78038e8-d70c-40ad-b9f6-2104ac1f4b3a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:55.376799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:55.388757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd412f-ba19-46a9-a882-bb43c46d8daf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:55.453816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38c2f3-c552-4646-8310-a5945291b385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:55.455329 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38c2f3-c552-4646-8310-a5945291b385 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.455807 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38c2f3-c552-4646-8310-a5945291b385 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38c2f3-c552-4646-8310-a5945291b385 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.466484 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38c2f3-c552-4646-8310-a5945291b385 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:55.468010 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:55.469401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38c2f3-c552-4646-8310-a5945291b385 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:55.470833 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 184/351] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:50:55 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:55.471775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38c2f3-c552-4646-8310-a5945291b385 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:55.488664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78038e8-d70c-40ad-b9f6-2104ac1f4b3a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:55.541624 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a5593-96b3-421b-af1c-005d54bb3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:55.543282 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a5593-96b3-421b-af1c-005d54bb3754 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.543449 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a5593-96b3-421b-af1c-005d54bb3754 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.543935 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a5593-96b3-421b-af1c-005d54bb3754 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.563483 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a5593-96b3-421b-af1c-005d54bb3754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3f7a5593-96b3-421b-af1c-005d54bb3754 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:55.574581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:55.575604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:55.576716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 168/352] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:50:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:55.577626 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81351a8-f808-4982-a548-43b3b7cd465c None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.577626 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81351a8-f808-4982-a548-43b3b7cd465c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.577925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81351a8-f808-4982-a548-43b3b7cd465c None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.579200 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 169/353] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:55.580899 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81351a8-f808-4982-a548-43b3b7cd465c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:55.640549 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38c2f3-c552-4646-8310-a5945291b385 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:55.679402 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3a3ed-3e05-4458-be1a-91a355811faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:55.680178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a3ed-3e05-4458-be1a-91a355811faa None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.680496 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a3ed-3e05-4458-be1a-91a355811faa None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.680806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a3ed-3e05-4458-be1a-91a355811faa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.695620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3a3ed-3e05-4458-be1a-91a355811faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-afe3a3ed-3e05-4458-be1a-91a355811faa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:55.703613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:55.704690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:50:55.705511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:55.706152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 185/354] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:55.707141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3a3ed-3e05-4458-be1a-91a355811faa None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:55.727858 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:55.736092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81351a8-f808-4982-a548-43b3b7cd465c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:55.786853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bdd01a-7dec-4229-b4a5-9d19ec1de1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:55.787738 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdd01a-7dec-4229-b4a5-9d19ec1de1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.788027 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdd01a-7dec-4229-b4a5-9d19ec1de1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.788359 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bdd01a-7dec-4229-b4a5-9d19ec1de1c1 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.789609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bdd01a-7dec-4229-b4a5-9d19ec1de1c1 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:55.791070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bdd01a-7dec-4229-b4a5-9d19ec1de1c1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:55.792941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 170/355] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:50:55 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:55.793565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bdd01a-7dec-4229-b4a5-9d19ec1de1c1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:55.822584 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:55.866787 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3a3ed-3e05-4458-be1a-91a355811faa None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:55.908299 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b6410-d465-4981-aca3-76af4f12571a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:55.908897 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6410-d465-4981-aca3-76af4f12571a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.909192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6410-d465-4981-aca3-76af4f12571a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.909470 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6410-d465-4981-aca3-76af4f12571a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.910310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b6410-d465-4981-aca3-76af4f12571a None admin] RBAC: Adding query filter params (group.id=c71c537edc9a4df28c3694808669589e, role.id=4498e695105347909981a684f613b6a2, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:55.912891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b6410-d465-4981-aca3-76af4f12571a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:55.916131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b6410-d465-4981-aca3-76af4f12571a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:55.922808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 186/356] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:50:55 2022] GET /identity/v3/role_assignments?group.id=c71c537edc9a4df28c3694808669589e&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=4498e695105347909981a684f613b6a2 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:55.925162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc055cd-b49a-461b-88bc-daa17a70df80 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.925162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc055cd-b49a-461b-88bc-daa17a70df80 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.925398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc055cd-b49a-461b-88bc-daa17a70df80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.940930 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bdd01a-7dec-4229-b4a5-9d19ec1de1c1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:55.946789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6fc055cd-b49a-461b-88bc-daa17a70df80 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:55.948043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6fc055cd-b49a-461b-88bc-daa17a70df80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:55.992400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493aebb-c0ee-451f-a049-efba59876931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:55.993603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493aebb-c0ee-451f-a049-efba59876931 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:55.995585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493aebb-c0ee-451f-a049-efba59876931 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:55.995585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493aebb-c0ee-451f-a049-efba59876931 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:55.996036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493aebb-c0ee-451f-a049-efba59876931 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:55.996421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493aebb-c0ee-451f-a049-efba59876931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:55.997900 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493aebb-c0ee-451f-a049-efba59876931 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:56.009594 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 171/357] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:50:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:56.010460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493aebb-c0ee-451f-a049-efba59876931 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:56.053164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 187/358] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:56.055378 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69d189-a2c5-4d7b-8716-5c78f40b4e9c None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:56.055672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69d189-a2c5-4d7b-8716-5c78f40b4e9c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:56.055950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69d189-a2c5-4d7b-8716-5c78f40b4e9c None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:56.057591 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 188/359] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:56.058122 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69d189-a2c5-4d7b-8716-5c78f40b4e9c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:56.075501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:56.153194 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:56.180067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493aebb-c0ee-451f-a049-efba59876931 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:56.209265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69d189-a2c5-4d7b-8716-5c78f40b4e9c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:56.228793 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f6803-4684-47ef-88ec-1367ba9f7a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:56.229638 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f6803-4684-47ef-88ec-1367ba9f7a4e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:56.230034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f6803-4684-47ef-88ec-1367ba9f7a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:56.230207 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f6803-4684-47ef-88ec-1367ba9f7a4e None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:56.230990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f6803-4684-47ef-88ec-1367ba9f7a4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:56.231304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f6803-4684-47ef-88ec-1367ba9f7a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:56.232752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f6803-4684-47ef-88ec-1367ba9f7a4e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:56.243101 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 172/360] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:56.243743 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f6803-4684-47ef-88ec-1367ba9f7a4e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:56.255395 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9c08d-3792-49f7-9b5f-a2bab3b02c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:56.256101 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9c08d-3792-49f7-9b5f-a2bab3b02c79 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:56.258326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9c08d-3792-49f7-9b5f-a2bab3b02c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:56.258647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9c08d-3792-49f7-9b5f-a2bab3b02c79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:56.259512 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9c08d-3792-49f7-9b5f-a2bab3b02c79 None admin] RBAC: Adding query filter params (group.id=c71c537edc9a4df28c3694808669589e, role.id=4498e695105347909981a684f613b6a2, scope.project.id=5589046ca8fb418da62e5cef63cedc9f) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:56.259828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9c08d-3792-49f7-9b5f-a2bab3b02c79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:56.262062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9c08d-3792-49f7-9b5f-a2bab3b02c79 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:56.271303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 189/361] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:50:56 2022] GET /identity/v3/role_assignments?group.id=c71c537edc9a4df28c3694808669589e&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=4498e695105347909981a684f613b6a2 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:56.273345 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c56e-0f3c-46f8-84d2-df73b5202140 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:56.273878 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c56e-0f3c-46f8-84d2-df73b5202140 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:56.274382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c56e-0f3c-46f8-84d2-df73b5202140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:56.294754 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2012c56e-0f3c-46f8-84d2-df73b5202140 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:56.296349 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2012c56e-0f3c-46f8-84d2-df73b5202140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:56.375910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f6803-4684-47ef-88ec-1367ba9f7a4e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:56.376860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:56.384075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 190/362] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:56.387020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c56e-0f3c-46f8-84d2-df73b5202140 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:56.442963 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d7b7c-3050-4267-bdc5-059cce4d812c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:56.443607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d7b7c-3050-4267-bdc5-059cce4d812c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:56.443973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d7b7c-3050-4267-bdc5-059cce4d812c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:56.444181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d7b7c-3050-4267-bdc5-059cce4d812c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:56.463059 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:56.521805 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c56e-0f3c-46f8-84d2-df73b5202140 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:56.550472 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d7b7c-3050-4267-bdc5-059cce4d812c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:56.553212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d7b7c-3050-4267-bdc5-059cce4d812c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:56.565428 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15503a60-f69a-43da-9a7d-5428ad140f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:56.566330 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15503a60-f69a-43da-9a7d-5428ad140f5f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:56.566646 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15503a60-f69a-43da-9a7d-5428ad140f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:56.567683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15503a60-f69a-43da-9a7d-5428ad140f5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:56.588031 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15503a60-f69a-43da-9a7d-5428ad140f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-15503a60-f69a-43da-9a7d-5428ad140f5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:56.597093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:56.598008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:50:56.599545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 191/363] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:50:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:56.603232 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15503a60-f69a-43da-9a7d-5428ad140f5f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:56.674779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:56.694646 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 173/364] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:56.697568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d7b7c-3050-4267-bdc5-059cce4d812c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:56.741961 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15503a60-f69a-43da-9a7d-5428ad140f5f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:56.782625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:56.808811 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcb9d1-5f2a-46a8-98f4-889b9ae1ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:56.820987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcb9d1-5f2a-46a8-98f4-889b9ae1ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:56.820987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcb9d1-5f2a-46a8-98f4-889b9ae1ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:56.820987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcb9d1-5f2a-46a8-98f4-889b9ae1ecf8 None admin] PATH_INFO: `/v3/users/b8f9b05586cb4d13999bfdce0c729b2b` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:56.820987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcb9d1-5f2a-46a8-98f4-889b9ae1ecf8 None admin] RBAC: Authorizing `identity:get_user(user_id=b8f9b05586cb4d13999bfdce0c729b2b)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:56.820987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcb9d1-5f2a-46a8-98f4-889b9ae1ecf8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:56.820987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 192/365] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:50:56 2022] GET /identity/v3/users/b8f9b05586cb4d13999bfdce0c729b2b => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:56.821672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcb9d1-5f2a-46a8-98f4-889b9ae1ecf8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:56.889140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d7b7c-3050-4267-bdc5-059cce4d812c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:56.933809 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a77077-a09f-4da4-803d-0c22618a72be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:56.934647 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a77077-a09f-4da4-803d-0c22618a72be None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:56.934936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a77077-a09f-4da4-803d-0c22618a72be None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:56.935216 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a77077-a09f-4da4-803d-0c22618a72be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:56.951845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a77077-a09f-4da4-803d-0c22618a72be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-40a77077-a09f-4da4-803d-0c22618a72be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:56.962272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:56.963580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:56.964416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 174/366] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:50:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:56.969543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a77077-a09f-4da4-803d-0c22618a72be None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:56.985448 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcb9d1-5f2a-46a8-98f4-889b9ae1ecf8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:57.021297 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:57.030817 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9199e97-11d7-416d-8940-feeecded3e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:57.031774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9199e97-11d7-416d-8940-feeecded3e6b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:57.032078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9199e97-11d7-416d-8940-feeecded3e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:57.033425 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9199e97-11d7-416d-8940-feeecded3e6b None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:57.034213 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9199e97-11d7-416d-8940-feeecded3e6b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:57.034530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9199e97-11d7-416d-8940-feeecded3e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:57.039185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9199e97-11d7-416d-8940-feeecded3e6b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:57.046947 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 193/367] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:50:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:57.047940 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9199e97-11d7-416d-8940-feeecded3e6b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:57.124494 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a77077-a09f-4da4-803d-0c22618a72be None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:57.128923 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:57.175481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85580a7e-eb29-4a17-9ed0-b53ddee99dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:57.178008 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85580a7e-eb29-4a17-9ed0-b53ddee99dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:57.178797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85580a7e-eb29-4a17-9ed0-b53ddee99dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:57.179275 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85580a7e-eb29-4a17-9ed0-b53ddee99dd3 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:57.180850 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85580a7e-eb29-4a17-9ed0-b53ddee99dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:57.182823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85580a7e-eb29-4a17-9ed0-b53ddee99dd3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:57.186632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 175/368] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:56 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:57.187676 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85580a7e-eb29-4a17-9ed0-b53ddee99dd3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:57.226999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9199e97-11d7-416d-8940-feeecded3e6b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:57.278404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a2400-9a04-466e-b927-535aacc5834a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:57.279441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a2400-9a04-466e-b927-535aacc5834a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:57.279733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a2400-9a04-466e-b927-535aacc5834a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:57.280014 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a2400-9a04-466e-b927-535aacc5834a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:57.281839 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a2400-9a04-466e-b927-535aacc5834a None admin] RBAC: Adding query filter params (role.id=8e0ef0ea0d26497db73b62d43d4fda22, user.id=b8f9b05586cb4d13999bfdce0c729b2b, scope.system=all) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:57.282163 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a2400-9a04-466e-b927-535aacc5834a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:57.284315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a2400-9a04-466e-b927-535aacc5834a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:57.307410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 194/369] 10.222.0.88 () {56 vars in 1308 bytes} [Thu Jun 16 14:50:57 2022] GET /identity/v3/role_assignments?user.id=b8f9b05586cb4d13999bfdce0c729b2b&scope.system=all&role.id=8e0ef0ea0d26497db73b62d43d4fda22 => generated 510 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:57.307410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a2400-9a04-466e-b927-535aacc5834a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:57.347882 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85580a7e-eb29-4a17-9ed0-b53ddee99dd3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:57.418979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:57.429860 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97eed6-f03b-4ede-be06-777797928a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:57.430968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97eed6-f03b-4ede-be06-777797928a54 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:57.432764 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97eed6-f03b-4ede-be06-777797928a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:57.433279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97eed6-f03b-4ede-be06-777797928a54 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:57.434160 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97eed6-f03b-4ede-be06-777797928a54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:57.437255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97eed6-f03b-4ede-be06-777797928a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:57.437255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97eed6-f03b-4ede-be06-777797928a54 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:57.452937 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 176/370] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:57.455030 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97eed6-f03b-4ede-be06-777797928a54 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:57.527165 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a2400-9a04-466e-b927-535aacc5834a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:57.531473 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:57.583560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca9e21-200f-423b-982a-60927b2dff8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:57.584259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca9e21-200f-423b-982a-60927b2dff8c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:57.584526 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca9e21-200f-423b-982a-60927b2dff8c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:57.584833 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca9e21-200f-423b-982a-60927b2dff8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:57.620942 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97eed6-f03b-4ede-be06-777797928a54 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:57.671556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6f289-4e01-4fc5-97f2-8388cdda450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:57.672465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6f289-4e01-4fc5-97f2-8388cdda450a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:57.672754 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6f289-4e01-4fc5-97f2-8388cdda450a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:57.673037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6f289-4e01-4fc5-97f2-8388cdda450a None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:57.675971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6f289-4e01-4fc5-97f2-8388cdda450a None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:57.679137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6f289-4e01-4fc5-97f2-8388cdda450a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:57.681144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca9e21-200f-423b-982a-60927b2dff8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:57.682651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca9e21-200f-423b-982a-60927b2dff8c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:57.686508 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 177/371] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:50:57 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:57.688502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6f289-4e01-4fc5-97f2-8388cdda450a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:57.756046 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:57.831857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6f289-4e01-4fc5-97f2-8388cdda450a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:57.833894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:57.834712 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 195/372] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:57.836601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca9e21-200f-423b-982a-60927b2dff8c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:57.878223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d4d30-68de-48a2-8ad0-8624e6fa9f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:57.878896 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d4d30-68de-48a2-8ad0-8624e6fa9f99 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:57.879186 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d4d30-68de-48a2-8ad0-8624e6fa9f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:57.880194 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d4d30-68de-48a2-8ad0-8624e6fa9f99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:57.962215 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d4d30-68de-48a2-8ad0-8624e6fa9f99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:57.963693 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d4d30-68de-48a2-8ad0-8624e6fa9f99 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:57.981975 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca9e21-200f-423b-982a-60927b2dff8c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:58.029149 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2bc8c-926e-48bc-9690-0e07fb37ff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:58.029968 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2bc8c-926e-48bc-9690-0e07fb37ff3c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:58.030331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2bc8c-926e-48bc-9690-0e07fb37ff3c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:58.030678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2bc8c-926e-48bc-9690-0e07fb37ff3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:58.031626 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2bc8c-926e-48bc-9690-0e07fb37ff3c None admin] RBAC: Adding query filter params (group.id=c71c537edc9a4df28c3694808669589e, role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:58.032002 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2bc8c-926e-48bc-9690-0e07fb37ff3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:58.033628 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2bc8c-926e-48bc-9690-0e07fb37ff3c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:58.041765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 196/373] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:50:57 2022] GET /identity/v3/role_assignments?group.id=c71c537edc9a4df28c3694808669589e&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:58.042661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2bc8c-926e-48bc-9690-0e07fb37ff3c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:58.065509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:58.124461 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 178/374] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:50:58.129535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d4d30-68de-48a2-8ad0-8624e6fa9f99 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:58.136162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:58.258736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2bc8c-926e-48bc-9690-0e07fb37ff3c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:58.313916 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9512c-bc34-42a5-bd4a-a1f4c70e7af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:58.314819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9512c-bc34-42a5-bd4a-a1f4c70e7af7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:58.315108 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9512c-bc34-42a5-bd4a-a1f4c70e7af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:58.315389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9512c-bc34-42a5-bd4a-a1f4c70e7af7 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:58.323610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9512c-bc34-42a5-bd4a-a1f4c70e7af7 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:58.325241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9512c-bc34-42a5-bd4a-a1f4c70e7af7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:58.327188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 197/375] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:58 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:58.351346 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9512c-bc34-42a5-bd4a-a1f4c70e7af7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:58.377812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d4d30-68de-48a2-8ad0-8624e6fa9f99 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:58.461094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5045f6-e8f5-4368-8ff3-655db99216d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:58.462393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5045f6-e8f5-4368-8ff3-655db99216d5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:58.462893 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5045f6-e8f5-4368-8ff3-655db99216d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:58.463475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5045f6-e8f5-4368-8ff3-655db99216d5 None admin] PATH_INFO: `/v3/projects/7d718157b3b949fc80592326cc9cfb5a` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:58.465147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5045f6-e8f5-4368-8ff3-655db99216d5 None admin] RBAC: Authorizing `identity:get_project(project_id=7d718157b3b949fc80592326cc9cfb5a)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:58.466915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5045f6-e8f5-4368-8ff3-655db99216d5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:58.468080 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:58.471919 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 179/376] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:58 2022] GET /identity/v3/projects/7d718157b3b949fc80592326cc9cfb5a => generated 344 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:58.477041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5045f6-e8f5-4368-8ff3-655db99216d5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:58.493639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9512c-bc34-42a5-bd4a-a1f4c70e7af7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:58.554956 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81a934-cd60-4ea3-9f54-dd3770a918e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:58.556307 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81a934-cd60-4ea3-9f54-dd3770a918e9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:58.556831 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81a934-cd60-4ea3-9f54-dd3770a918e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:58.557332 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81a934-cd60-4ea3-9f54-dd3770a918e9 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:58.560075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81a934-cd60-4ea3-9f54-dd3770a918e9 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:58.561914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81a934-cd60-4ea3-9f54-dd3770a918e9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:58.564131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 198/377] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:50:58 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:58.572369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81a934-cd60-4ea3-9f54-dd3770a918e9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:58.594012 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:58.639066 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5045f6-e8f5-4368-8ff3-655db99216d5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:58.699766 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc8335-ff94-4fcd-a827-8e936cbd868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:58.701800 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc8335-ff94-4fcd-a827-8e936cbd868e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:58.701800 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc8335-ff94-4fcd-a827-8e936cbd868e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:58.701800 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc8335-ff94-4fcd-a827-8e936cbd868e None admin] PATH_INFO: `/v3/groups/6a7775b234af40119804bd96800429f9` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:58.712205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc8335-ff94-4fcd-a827-8e936cbd868e None admin] RBAC: Authorizing `identity:get_group(group_id=6a7775b234af40119804bd96800429f9)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:58.713587 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc8335-ff94-4fcd-a827-8e936cbd868e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:58.720324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 180/378] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:50:58 2022] GET /identity/v3/groups/6a7775b234af40119804bd96800429f9 => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:50:58.721889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a93bb8-ac32-4b61-989e-c77b3ec4be3d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:58.722183 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a93bb8-ac32-4b61-989e-c77b3ec4be3d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:58.722469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a93bb8-ac32-4b61-989e-c77b3ec4be3d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:58.728250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 181/379] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:58.729901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a93bb8-ac32-4b61-989e-c77b3ec4be3d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:58.772810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81a934-cd60-4ea3-9f54-dd3770a918e9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:58.817201 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d3d8a-b5e1-4ca2-a022-e4a2149dbe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:58.818348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d3d8a-b5e1-4ca2-a022-e4a2149dbe89 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:58.818860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d3d8a-b5e1-4ca2-a022-e4a2149dbe89 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:58.819340 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d3d8a-b5e1-4ca2-a022-e4a2149dbe89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:58.820458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d3d8a-b5e1-4ca2-a022-e4a2149dbe89 None admin] RBAC: Adding query filter params (group.id=c71c537edc9a4df28c3694808669589e, role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=5589046ca8fb418da62e5cef63cedc9f) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:58.820989 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d3d8a-b5e1-4ca2-a022-e4a2149dbe89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:58.822708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d3d8a-b5e1-4ca2-a022-e4a2149dbe89 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:58.829586 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 199/380] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:50:58 2022] GET /identity/v3/role_assignments?group.id=c71c537edc9a4df28c3694808669589e&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:58.831289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acaac3-4d2f-4ffd-a25b-30d1c8b6a9be None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:58.831802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acaac3-4d2f-4ffd-a25b-30d1c8b6a9be None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:58.832362 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9acaac3-4d2f-4ffd-a25b-30d1c8b6a9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:58.851563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e9acaac3-4d2f-4ffd-a25b-30d1c8b6a9be None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:58.853494 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e9acaac3-4d2f-4ffd-a25b-30d1c8b6a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:58.853630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a93bb8-ac32-4b61-989e-c77b3ec4be3d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:58.873527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:58.894462 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd51b-b62f-4df1-ad77-35fee41d876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:58.895135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd51b-b62f-4df1-ad77-35fee41d876d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:58.895428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd51b-b62f-4df1-ad77-35fee41d876d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:58.895710 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd51b-b62f-4df1-ad77-35fee41d876d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:58.896738 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cd51b-b62f-4df1-ad77-35fee41d876d None admin] RBAC: Adding query filter params (group.id=6a7775b234af40119804bd96800429f9, role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7d718157b3b949fc80592326cc9cfb5a) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:58.897112 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cd51b-b62f-4df1-ad77-35fee41d876d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:58.898636 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cd51b-b62f-4df1-ad77-35fee41d876d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:58.906776 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:58.912428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 182/381] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:50:58 2022] GET /identity/v3/role_assignments?group.id=6a7775b234af40119804bd96800429f9&scope.project.id=7d718157b3b949fc80592326cc9cfb5a&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:58.949662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 200/382] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:50:58.975242 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd51b-b62f-4df1-ad77-35fee41d876d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:59.134647 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd51b-b62f-4df1-ad77-35fee41d876d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:59.178206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a674cb8-e1dc-4e42-9f6c-a8c7f8a8e182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:59.178946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a674cb8-e1dc-4e42-9f6c-a8c7f8a8e182 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:59.179232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a674cb8-e1dc-4e42-9f6c-a8c7f8a8e182 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:59.179512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a674cb8-e1dc-4e42-9f6c-a8c7f8a8e182 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:59.192625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a674cb8-e1dc-4e42-9f6c-a8c7f8a8e182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3a674cb8-e1dc-4e42-9f6c-a8c7f8a8e182 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:50:59.201099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:50:59.202098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:50:59.202951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 183/383] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:50:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:50:59.209120 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acaac3-4d2f-4ffd-a25b-30d1c8b6a9be None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:59.358430 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9acaac3-4d2f-4ffd-a25b-30d1c8b6a9be None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:59.403930 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f1c42-425d-43ec-9507-0447ffe41928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:59.404639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f1c42-425d-43ec-9507-0447ffe41928 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:59.404935 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f1c42-425d-43ec-9507-0447ffe41928 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:59.405217 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f1c42-425d-43ec-9507-0447ffe41928 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:59.405977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f1c42-425d-43ec-9507-0447ffe41928 None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:50:59.406291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f1c42-425d-43ec-9507-0447ffe41928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:59.407810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f1c42-425d-43ec-9507-0447ffe41928 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:59.420266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 201/384] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:50:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:59.429426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a674cb8-e1dc-4e42-9f6c-a8c7f8a8e182 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:59.528362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:59.589271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a674cb8-e1dc-4e42-9f6c-a8c7f8a8e182 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:59.638509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab621c1-2648-48fa-a02b-88341fc05cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:59.639726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab621c1-2648-48fa-a02b-88341fc05cda None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:59.640299 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab621c1-2648-48fa-a02b-88341fc05cda None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:59.640471 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab621c1-2648-48fa-a02b-88341fc05cda None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:59.641719 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab621c1-2648-48fa-a02b-88341fc05cda None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:59.643189 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab621c1-2648-48fa-a02b-88341fc05cda None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:59.649456 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 184/385] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:50:59 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:59.653685 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f1c42-425d-43ec-9507-0447ffe41928 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:59.662231 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:59.762373 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106aadb-fada-4362-9803-e1f462cd799b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:59.763037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106aadb-fada-4362-9803-e1f462cd799b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:59.763530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106aadb-fada-4362-9803-e1f462cd799b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:59.765324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 185/386] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:59.778502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf5953-ae03-44e7-a9ef-19318a902f01 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:59.778821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf5953-ae03-44e7-a9ef-19318a902f01 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:59.779112 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf5953-ae03-44e7-a9ef-19318a902f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:59.814769 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f1c42-425d-43ec-9507-0447ffe41928 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:59.816197 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-eecf5953-ae03-44e7-a9ef-19318a902f01 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:59.820270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-eecf5953-ae03-44e7-a9ef-19318a902f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:59.898724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e839ce-a959-43cc-aa41-16867f0fdd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:59.899935 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e839ce-a959-43cc-aa41-16867f0fdd0a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:59.900523 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e839ce-a959-43cc-aa41-16867f0fdd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:59.900957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e839ce-a959-43cc-aa41-16867f0fdd0a None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:59.902471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e839ce-a959-43cc-aa41-16867f0fdd0a None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:59.904093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e839ce-a959-43cc-aa41-16867f0fdd0a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:59.906350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 202/387] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:50:59 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:50:59.913773 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e839ce-a959-43cc-aa41-16867f0fdd0a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:59.988288 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 186/388] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:00.047990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf5953-ae03-44e7-a9ef-19318a902f01 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:00.053528 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:00.105062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e839ce-a959-43cc-aa41-16867f0fdd0a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:00.120594 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:00.175613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1260a6-0a2c-4c9b-9bc4-816567b348c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:00.177132 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1260a6-0a2c-4c9b-9bc4-816567b348c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:00.177627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1260a6-0a2c-4c9b-9bc4-816567b348c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:00.178141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1260a6-0a2c-4c9b-9bc4-816567b348c7 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/groups/c71c537edc9a4df28c3694808669589e/roles/4498e695105347909981a684f613b6a2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:00.185005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1260a6-0a2c-4c9b-9bc4-816567b348c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c4956db4fa4484cbedbb6ce367d55de, group_id=c71c537edc9a4df28c3694808669589e, role_id=4498e695105347909981a684f613b6a2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:00.185404 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf5953-ae03-44e7-a9ef-19318a902f01 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:00.186959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1260a6-0a2c-4c9b-9bc4-816567b348c7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:00.223362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb649b-a4b4-46c2-a7aa-20ea5a860e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:00.224278 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb649b-a4b4-46c2-a7aa-20ea5a860e7f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:00.224573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb649b-a4b4-46c2-a7aa-20ea5a860e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:00.225067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb649b-a4b4-46c2-a7aa-20ea5a860e7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:00.236438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb649b-a4b4-46c2-a7aa-20ea5a860e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-82cb649b-a4b4-46c2-a7aa-20ea5a860e7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:00.245149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:00.249614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:00.250543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 187/389] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:00.256425 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb649b-a4b4-46c2-a7aa-20ea5a860e7f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:00.283914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 203/390] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 14:50:59 2022] PUT /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/groups/c71c537edc9a4df28c3694808669589e/roles/4498e695105347909981a684f613b6a2 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:00.380324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb649b-a4b4-46c2-a7aa-20ea5a860e7f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:00.410367 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:00.435051 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de937fe0-e2a7-4f4d-859a-648406933451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:00.435741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de937fe0-e2a7-4f4d-859a-648406933451 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:00.436026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de937fe0-e2a7-4f4d-859a-648406933451 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:00.436341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de937fe0-e2a7-4f4d-859a-648406933451 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:00.437121 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de937fe0-e2a7-4f4d-859a-648406933451 None admin] RBAC: Adding query filter params (name=member) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:00.437434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de937fe0-e2a7-4f4d-859a-648406933451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:00.438967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de937fe0-e2a7-4f4d-859a-648406933451 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:00.450526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 188/391] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:51:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:00.457623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1260a6-0a2c-4c9b-9bc4-816567b348c7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:00.624753 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1260a6-0a2c-4c9b-9bc4-816567b348c7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:00.664488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:00.674652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132a432-3563-4848-87a5-2df02858ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:00.679350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132a432-3563-4848-87a5-2df02858ad57 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:00.679350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132a432-3563-4848-87a5-2df02858ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:00.679350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132a432-3563-4848-87a5-2df02858ad57 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:00.679350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132a432-3563-4848-87a5-2df02858ad57 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:00.679903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132a432-3563-4848-87a5-2df02858ad57 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:00.686856 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 204/392] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:51:00 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:00.688650 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de937fe0-e2a7-4f4d-859a-648406933451 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:00.862050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de937fe0-e2a7-4f4d-859a-648406933451 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:00.915279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799c299-009e-4e9b-9eff-10e78345d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:00.916249 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799c299-009e-4e9b-9eff-10e78345d908 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:00.916594 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799c299-009e-4e9b-9eff-10e78345d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:00.916907 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799c299-009e-4e9b-9eff-10e78345d908 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:00.918130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799c299-009e-4e9b-9eff-10e78345d908 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:00.919627 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799c299-009e-4e9b-9eff-10e78345d908 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:00.922004 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 189/393] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:51:00 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:00.928991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132a432-3563-4848-87a5-2df02858ad57 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:01.087008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132a432-3563-4848-87a5-2df02858ad57 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:01.131755 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb49a9-10d4-438d-96ff-77ee0d1a8ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:01.133296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb49a9-10d4-438d-96ff-77ee0d1a8ec9 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:01.133783 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb49a9-10d4-438d-96ff-77ee0d1a8ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:01.134254 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb49a9-10d4-438d-96ff-77ee0d1a8ec9 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f/groups/c71c537edc9a4df28c3694808669589e/roles/4498e695105347909981a684f613b6a2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:01.141718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb49a9-10d4-438d-96ff-77ee0d1a8ec9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5589046ca8fb418da62e5cef63cedc9f, group_id=c71c537edc9a4df28c3694808669589e, role_id=4498e695105347909981a684f613b6a2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:01.143591 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb49a9-10d4-438d-96ff-77ee0d1a8ec9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:01.219666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a308440-6f56-4006-9193-13ad9ef63266 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:01.219910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a308440-6f56-4006-9193-13ad9ef63266 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:01.220165 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a308440-6f56-4006-9193-13ad9ef63266 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:01.221928 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 190/394] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:01.227859 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc64ad1-8fa2-4fd4-b70c-c79a8cce121e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:01.228159 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc64ad1-8fa2-4fd4-b70c-c79a8cce121e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:01.228669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc64ad1-8fa2-4fd4-b70c-c79a8cce121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:01.249773 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 205/395] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 14:51:00 2022] PUT /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f/groups/c71c537edc9a4df28c3694808669589e/roles/4498e695105347909981a684f613b6a2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:01.257268 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cfc64ad1-8fa2-4fd4-b70c-c79a8cce121e None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:01.257268 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cfc64ad1-8fa2-4fd4-b70c-c79a8cce121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:01.319757 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:01.351008 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 191/396] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:51:01.375592 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb49a9-10d4-438d-96ff-77ee0d1a8ec9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:01.461041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:01.517766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb49a9-10d4-438d-96ff-77ee0d1a8ec9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:01.565044 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00716f8b-c2c8-414d-8cff-1b5068f0c6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:01.565889 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00716f8b-c2c8-414d-8cff-1b5068f0c6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:01.566176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00716f8b-c2c8-414d-8cff-1b5068f0c6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:01.566454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00716f8b-c2c8-414d-8cff-1b5068f0c6f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:01.585822 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00716f8b-c2c8-414d-8cff-1b5068f0c6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-00716f8b-c2c8-414d-8cff-1b5068f0c6f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:01.594967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:01.596434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:01.598113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 206/397] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:51:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:01.605829 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc64ad1-8fa2-4fd4-b70c-c79a8cce121e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:01.687795 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99659917-89f2-451a-9bdf-f0f20f05eb06 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:01.688102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99659917-89f2-451a-9bdf-f0f20f05eb06 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:01.688367 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99659917-89f2-451a-9bdf-f0f20f05eb06 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:01.690667 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 207/398] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:01.698298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dab01-1170-4879-a498-823a055f5810 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:01.698589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dab01-1170-4879-a498-823a055f5810 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:01.698868 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dab01-1170-4879-a498-823a055f5810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:01.736357 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b27dab01-1170-4879-a498-823a055f5810 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:01.737574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b27dab01-1170-4879-a498-823a055f5810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:01.797934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc64ad1-8fa2-4fd4-b70c-c79a8cce121e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:01.852364 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 208/399] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:51:01.860737 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9837e-5169-49d6-83c2-42b9e98bc715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:01.861793 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9837e-5169-49d6-83c2-42b9e98bc715 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:01.862291 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9837e-5169-49d6-83c2-42b9e98bc715 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:01.862768 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9837e-5169-49d6-83c2-42b9e98bc715 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:01.864416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9837e-5169-49d6-83c2-42b9e98bc715 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:01.865096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9837e-5169-49d6-83c2-42b9e98bc715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:01.866749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9837e-5169-49d6-83c2-42b9e98bc715 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:01.876671 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 192/400] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:01.884292 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dab01-1170-4879-a498-823a055f5810 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:01.892470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9837e-5169-49d6-83c2-42b9e98bc715 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:01.953026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:02.010595 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:02.030732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9837e-5169-49d6-83c2-42b9e98bc715 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:02.079436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31335c1c-ec8f-4352-94c2-cce644f1338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:02.080207 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31335c1c-ec8f-4352-94c2-cce644f1338f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.080536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31335c1c-ec8f-4352-94c2-cce644f1338f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.080827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31335c1c-ec8f-4352-94c2-cce644f1338f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.083979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dab01-1170-4879-a498-823a055f5810 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:02.096409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31335c1c-ec8f-4352-94c2-cce644f1338f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-31335c1c-ec8f-4352-94c2-cce644f1338f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:02.107450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:02.108523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:02.110135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:02.114582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 193/401] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:51:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:02.116482 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefb852-2ad2-47c4-901a-c425c51377a9 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.116774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefb852-2ad2-47c4-901a-c425c51377a9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.117078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefb852-2ad2-47c4-901a-c425c51377a9 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.118925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 194/402] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:02.120166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefb852-2ad2-47c4-901a-c425c51377a9 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:02.134681 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754382e-6f1f-4927-b0f2-a3e70f20b100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:02.135940 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754382e-6f1f-4927-b0f2-a3e70f20b100 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.136487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754382e-6f1f-4927-b0f2-a3e70f20b100 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.136970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754382e-6f1f-4927-b0f2-a3e70f20b100 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.138438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754382e-6f1f-4927-b0f2-a3e70f20b100 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:02.140177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754382e-6f1f-4927-b0f2-a3e70f20b100 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:02.142671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 209/403] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:51:01 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:02.144313 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6009c-9b52-43b9-8e4b-8ddb07270ef2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.145368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6009c-9b52-43b9-8e4b-8ddb07270ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.145901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6009c-9b52-43b9-8e4b-8ddb07270ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.165531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4dd6009c-9b52-43b9-8e4b-8ddb07270ef2 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:02.167135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4dd6009c-9b52-43b9-8e4b-8ddb07270ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:02.248206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 210/404] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:02.249607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6009c-9b52-43b9-8e4b-8ddb07270ef2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:02.251296 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:02.256939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefb852-2ad2-47c4-901a-c425c51377a9 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:02.295954 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d2adb-3e91-44c2-9f75-272837903f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:02.297134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d2adb-3e91-44c2-9f75-272837903f39 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.297724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d2adb-3e91-44c2-9f75-272837903f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.298265 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d2adb-3e91-44c2-9f75-272837903f39 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.300150 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d2adb-3e91-44c2-9f75-272837903f39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:02.301108 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d2adb-3e91-44c2-9f75-272837903f39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:02.304181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d2adb-3e91-44c2-9f75-272837903f39 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:02.315896 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:02.319363 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 195/405] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:02.320613 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d2adb-3e91-44c2-9f75-272837903f39 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:02.399036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6009c-9b52-43b9-8e4b-8ddb07270ef2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:02.449368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b699e-5065-49d3-9986-1dd2c072a726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:02.450678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b699e-5065-49d3-9986-1dd2c072a726 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.450972 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b699e-5065-49d3-9986-1dd2c072a726 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.451251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b699e-5065-49d3-9986-1dd2c072a726 None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.452771 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b699e-5065-49d3-9986-1dd2c072a726 None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:02.454908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b699e-5065-49d3-9986-1dd2c072a726 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:02.456983 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 211/406] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:51:02 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:02.458767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b699e-5065-49d3-9986-1dd2c072a726 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:02.459551 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d2adb-3e91-44c2-9f75-272837903f39 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:02.510657 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff3077-e61b-415f-b4ea-2b05abe389c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:02.511857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff3077-e61b-415f-b4ea-2b05abe389c5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.512388 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff3077-e61b-415f-b4ea-2b05abe389c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.512981 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff3077-e61b-415f-b4ea-2b05abe389c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.529010 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff3077-e61b-415f-b4ea-2b05abe389c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a1ff3077-e61b-415f-b4ea-2b05abe389c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:02.538266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:02.540845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:02.541895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 196/407] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:02.542763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff3077-e61b-415f-b4ea-2b05abe389c5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:02.576805 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:02.605727 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b699e-5065-49d3-9986-1dd2c072a726 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:02.626435 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:02.656748 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dda425-f070-41fc-9cb4-e345946758c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:02.657546 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dda425-f070-41fc-9cb4-e345946758c0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.657832 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dda425-f070-41fc-9cb4-e345946758c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.658109 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dda425-f070-41fc-9cb4-e345946758c0 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.659462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dda425-f070-41fc-9cb4-e345946758c0 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:02.661789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dda425-f070-41fc-9cb4-e345946758c0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:02.668341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 212/408] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:51:02 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:02.670066 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dda425-f070-41fc-9cb4-e345946758c0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:02.707140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff3077-e61b-415f-b4ea-2b05abe389c5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:02.750400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1457-344d-4f69-85c1-b2c4055bc57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:02.751364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1457-344d-4f69-85c1-b2c4055bc57a None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.751670 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1457-344d-4f69-85c1-b2c4055bc57a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.751950 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1457-344d-4f69-85c1-b2c4055bc57a None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/groups/c71c537edc9a4df28c3694808669589e/roles/f4c6e6a3b05a42abb56447797f7ade15` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.754959 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b1457-344d-4f69-85c1-b2c4055bc57a None admin] RBAC: Authorizing `identity:create_grant(project_id=7c4956db4fa4484cbedbb6ce367d55de, group_id=c71c537edc9a4df28c3694808669589e, role_id=f4c6e6a3b05a42abb56447797f7ade15)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:02.756370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b1457-344d-4f69-85c1-b2c4055bc57a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:02.801829 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dda425-f070-41fc-9cb4-e345946758c0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:02.840712 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fe21f-cbbe-4158-a1a8-9df14349d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:02.841510 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fe21f-cbbe-4158-a1a8-9df14349d37c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.841807 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fe21f-cbbe-4158-a1a8-9df14349d37c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.842090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fe21f-cbbe-4158-a1a8-9df14349d37c None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.843193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80fe21f-cbbe-4158-a1a8-9df14349d37c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:02.843193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80fe21f-cbbe-4158-a1a8-9df14349d37c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:02.844877 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80fe21f-cbbe-4158-a1a8-9df14349d37c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:02.857598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 197/409] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 14:51:02 2022] PUT /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de/groups/c71c537edc9a4df28c3694808669589e/roles/f4c6e6a3b05a42abb56447797f7ade15 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:02.858474 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1457-344d-4f69-85c1-b2c4055bc57a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:02.862082 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 213/410] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:02.873769 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fe21f-cbbe-4158-a1a8-9df14349d37c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:02.999754 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:03.026001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:03.028040 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1457-344d-4f69-85c1-b2c4055bc57a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:03.074296 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f3160-6b31-4fb9-b557-eab6062c45f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:03.075785 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f3160-6b31-4fb9-b557-eab6062c45f0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.076458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f3160-6b31-4fb9-b557-eab6062c45f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.076985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f3160-6b31-4fb9-b557-eab6062c45f0 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.079046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f3160-6b31-4fb9-b557-eab6062c45f0 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:03.080904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f3160-6b31-4fb9-b557-eab6062c45f0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:03.088478 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 198/411] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:51:02 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:03.092270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f3160-6b31-4fb9-b557-eab6062c45f0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:03.137612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fe21f-cbbe-4158-a1a8-9df14349d37c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:03.184673 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ad01c-8ec5-4651-8cdb-72ab8bb6a946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:03.186120 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ad01c-8ec5-4651-8cdb-72ab8bb6a946 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.186411 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ad01c-8ec5-4651-8cdb-72ab8bb6a946 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.186693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ad01c-8ec5-4651-8cdb-72ab8bb6a946 None admin] PATH_INFO: `/v3/groups/6a7775b234af40119804bd96800429f9` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.187898 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ad01c-8ec5-4651-8cdb-72ab8bb6a946 None admin] RBAC: Authorizing `identity:get_group(group_id=6a7775b234af40119804bd96800429f9)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:03.190104 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ad01c-8ec5-4651-8cdb-72ab8bb6a946 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:03.192515 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 214/412] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:51:02 2022] GET /identity/v3/groups/6a7775b234af40119804bd96800429f9 => generated 229 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:03.193762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993780f-8bbf-4527-b089-b183af1fd949 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.194054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993780f-8bbf-4527-b089-b183af1fd949 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.194337 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993780f-8bbf-4527-b089-b183af1fd949 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.195865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 215/413] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:03.199678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993780f-8bbf-4527-b089-b183af1fd949 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:03.237047 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f3160-6b31-4fb9-b557-eab6062c45f0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:03.282658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e10ae-11a3-4054-af5e-b128855714ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:03.283740 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e10ae-11a3-4054-af5e-b128855714ae None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.284076 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e10ae-11a3-4054-af5e-b128855714ae None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.284435 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e10ae-11a3-4054-af5e-b128855714ae None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f/groups/c71c537edc9a4df28c3694808669589e/roles/f4c6e6a3b05a42abb56447797f7ade15` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.287494 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e10ae-11a3-4054-af5e-b128855714ae None admin] RBAC: Authorizing `identity:create_grant(project_id=5589046ca8fb418da62e5cef63cedc9f, group_id=c71c537edc9a4df28c3694808669589e, role_id=f4c6e6a3b05a42abb56447797f7ade15)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:03.289142 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e10ae-11a3-4054-af5e-b128855714ae None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:03.332142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993780f-8bbf-4527-b089-b183af1fd949 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:03.351055 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:03.370779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 199/414] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 14:51:03 2022] PUT /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f/groups/c71c537edc9a4df28c3694808669589e/roles/f4c6e6a3b05a42abb56447797f7ade15 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:51:03.372967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab768f7-9e48-40ba-b0a1-dbb3a2910d8f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.373572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab768f7-9e48-40ba-b0a1-dbb3a2910d8f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.374235 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab768f7-9e48-40ba-b0a1-dbb3a2910d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.375076 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e1346-7d66-440c-a8b7-a4b335296ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:03.375968 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e1346-7d66-440c-a8b7-a4b335296ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.376286 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e1346-7d66-440c-a8b7-a4b335296ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.376566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e1346-7d66-440c-a8b7-a4b335296ac3 None admin] PATH_INFO: `/v3/projects/7d718157b3b949fc80592326cc9cfb5a` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.377845 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e1346-7d66-440c-a8b7-a4b335296ac3 None admin] RBAC: Authorizing `identity:get_project(project_id=7d718157b3b949fc80592326cc9cfb5a)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:03.379616 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e1346-7d66-440c-a8b7-a4b335296ac3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:03.381714 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 216/415] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:51:03 2022] GET /identity/v3/projects/7d718157b3b949fc80592326cc9cfb5a => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:03.387620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e1346-7d66-440c-a8b7-a4b335296ac3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:03.400991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-aab768f7-9e48-40ba-b0a1-dbb3a2910d8f None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:03.403534 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-aab768f7-9e48-40ba-b0a1-dbb3a2910d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:03.407296 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:03.511240 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 200/416] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:03.544660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e1346-7d66-440c-a8b7-a4b335296ac3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:03.549872 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab768f7-9e48-40ba-b0a1-dbb3a2910d8f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:03.631521 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659b58c-84ff-48ee-ace1-d91217b7c4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:03.632448 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659b58c-84ff-48ee-ace1-d91217b7c4ee None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.632738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659b58c-84ff-48ee-ace1-d91217b7c4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.633275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659b58c-84ff-48ee-ace1-d91217b7c4ee None admin] PATH_INFO: `/v3/projects/7d718157b3b949fc80592326cc9cfb5a/groups/6a7775b234af40119804bd96800429f9/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.635317 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659b58c-84ff-48ee-ace1-d91217b7c4ee None admin] RBAC: Authorizing `identity:create_grant(project_id=7d718157b3b949fc80592326cc9cfb5a, group_id=6a7775b234af40119804bd96800429f9, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:03.636811 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659b58c-84ff-48ee-ace1-d91217b7c4ee None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:03.688887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab768f7-9e48-40ba-b0a1-dbb3a2910d8f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:03.727204 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f220c-fe61-47fb-b156-090ac917f94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:03.728523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f220c-fe61-47fb-b156-090ac917f94c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.729008 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f220c-fe61-47fb-b156-090ac917f94c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.729478 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f220c-fe61-47fb-b156-090ac917f94c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.739123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:03.743371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f220c-fe61-47fb-b156-090ac917f94c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3c8f220c-fe61-47fb-b156-090ac917f94c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:03.752318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:03.753398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:51:03.754978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:03.765252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 217/417] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 14:51:03 2022] PUT /identity/v3/projects/7d718157b3b949fc80592326cc9cfb5a/groups/6a7775b234af40119804bd96800429f9/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:51:03.768072 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 201/418] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:03.773366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659b58c-84ff-48ee-ace1-d91217b7c4ee None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:03.825133 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5b1a7-710c-4810-b0b2-1b0a9723b9d2 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.825341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5b1a7-710c-4810-b0b2-1b0a9723b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.825627 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5b1a7-710c-4810-b0b2-1b0a9723b9d2 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.831303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:03.831486 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 202/419] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:51:03.833827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be398e-967d-447d-8937-076c3183e777 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.834124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be398e-967d-447d-8937-076c3183e777 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.834406 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be398e-967d-447d-8937-076c3183e777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.862698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-23be398e-967d-447d-8937-076c3183e777 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:03.864823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-23be398e-967d-447d-8937-076c3183e777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:03.950837 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659b58c-84ff-48ee-ace1-d91217b7c4ee None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:03.995599 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44964a-cbba-4676-901c-4e05c32a7fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:03.996319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44964a-cbba-4676-901c-4e05c32a7fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.996615 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44964a-cbba-4676-901c-4e05c32a7fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.996899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44964a-cbba-4676-901c-4e05c32a7fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.997775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44964a-cbba-4676-901c-4e05c32a7fa7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:03.997973 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44964a-cbba-4676-901c-4e05c32a7fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:03.998166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 203/420] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:51:03.999446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44964a-cbba-4676-901c-4e05c32a7fa7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:04.009767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 218/421] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:51:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:04.015188 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be398e-967d-447d-8937-076c3183e777 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:04.023380 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44964a-cbba-4676-901c-4e05c32a7fa7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:04.155574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44964a-cbba-4676-901c-4e05c32a7fa7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:04.159033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:04.196915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15011b2-3055-45d7-957c-4ff7904b2716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:04.198969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15011b2-3055-45d7-957c-4ff7904b2716 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:04.199601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15011b2-3055-45d7-957c-4ff7904b2716 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:04.200078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15011b2-3055-45d7-957c-4ff7904b2716 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:04.214033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15011b2-3055-45d7-957c-4ff7904b2716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e15011b2-3055-45d7-957c-4ff7904b2716 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:04.223365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:04.224796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:51:04.225734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:04.228707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 219/422] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:04.237319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15011b2-3055-45d7-957c-4ff7904b2716 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:04.253570 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be398e-967d-447d-8937-076c3183e777 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:04.300552 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:04.325436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856278d8-847c-4d91-89f2-75494b9e0f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:04.326144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856278d8-847c-4d91-89f2-75494b9e0f59 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:04.326436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856278d8-847c-4d91-89f2-75494b9e0f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:04.326715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856278d8-847c-4d91-89f2-75494b9e0f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:04.391363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15011b2-3055-45d7-957c-4ff7904b2716 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:04.426955 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae27a3-b627-42d9-b0b2-9c2ca446614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:04.427984 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae27a3-b627-42d9-b0b2-9c2ca446614b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:04.428517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae27a3-b627-42d9-b0b2-9c2ca446614b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:04.429027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae27a3-b627-42d9-b0b2-9c2ca446614b None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:04.429987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae27a3-b627-42d9-b0b2-9c2ca446614b None admin] RBAC: Adding query filter params (name=member) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:04.430516 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae27a3-b627-42d9-b0b2-9c2ca446614b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:04.432123 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae27a3-b627-42d9-b0b2-9c2ca446614b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:04.443402 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 220/423] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:51:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:04.451190 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae27a3-b627-42d9-b0b2-9c2ca446614b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:04.464669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856278d8-847c-4d91-89f2-75494b9e0f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:04.466503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856278d8-847c-4d91-89f2-75494b9e0f59 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:04.524592 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:04.590796 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 204/424] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:04.596299 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856278d8-847c-4d91-89f2-75494b9e0f59 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:04.656964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:04.713517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae27a3-b627-42d9-b0b2-9c2ca446614b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:04.796282 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856278d8-847c-4d91-89f2-75494b9e0f59 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:04.820954 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f446f3c-494d-4989-a2fd-2e669e8dc84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:04.821628 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f446f3c-494d-4989-a2fd-2e669e8dc84a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:04.821920 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f446f3c-494d-4989-a2fd-2e669e8dc84a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:04.822204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f446f3c-494d-4989-a2fd-2e669e8dc84a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:04.859704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bb298-179d-4b12-bb6b-e4bd1eb2e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:04.861206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bb298-179d-4b12-bb6b-e4bd1eb2e74d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:04.861694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bb298-179d-4b12-bb6b-e4bd1eb2e74d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:04.862183 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bb298-179d-4b12-bb6b-e4bd1eb2e74d None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:04.864316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bb298-179d-4b12-bb6b-e4bd1eb2e74d None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:04.867080 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bb298-179d-4b12-bb6b-e4bd1eb2e74d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:04.871203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 205/425] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:51:04 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:04.876592 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bb298-179d-4b12-bb6b-e4bd1eb2e74d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:05.000157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bb298-179d-4b12-bb6b-e4bd1eb2e74d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:05.012656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:05.033203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f446f3c-494d-4989-a2fd-2e669e8dc84a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:05.034722 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f446f3c-494d-4989-a2fd-2e669e8dc84a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:05.041032 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11b507-74d3-4ed8-9036-b6b9faf0ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:05.042835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11b507-74d3-4ed8-9036-b6b9faf0ef02 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:05.043136 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11b507-74d3-4ed8-9036-b6b9faf0ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:05.043426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11b507-74d3-4ed8-9036-b6b9faf0ef02 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:05.044712 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11b507-74d3-4ed8-9036-b6b9faf0ef02 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:05.046269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11b507-74d3-4ed8-9036-b6b9faf0ef02 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:05.048427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 206/426] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:51:04 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:05.059712 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11b507-74d3-4ed8-9036-b6b9faf0ef02 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:05.094919 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:05.250792 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 221/427] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 800 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:05.257136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f446f3c-494d-4989-a2fd-2e669e8dc84a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:05.277890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11b507-74d3-4ed8-9036-b6b9faf0ef02 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:05.326724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007bacd-16b8-4821-9255-d138ab898bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:05.327854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007bacd-16b8-4821-9255-d138ab898bdf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:05.329374 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007bacd-16b8-4821-9255-d138ab898bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:05.329856 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007bacd-16b8-4821-9255-d138ab898bdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:05.333164 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007bacd-16b8-4821-9255-d138ab898bdf None admin] RBAC: Adding query filter params (group.id=c71c537edc9a4df28c3694808669589e, role.id=4498e695105347909981a684f613b6a2, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:05.333739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007bacd-16b8-4821-9255-d138ab898bdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:05.335437 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007bacd-16b8-4821-9255-d138ab898bdf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:05.352878 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 207/428] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:51:05 2022] GET /identity/v3/role_assignments?group.id=c71c537edc9a4df28c3694808669589e&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=4498e695105347909981a684f613b6a2 => generated 611 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:05.447000 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f446f3c-494d-4989-a2fd-2e669e8dc84a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:05.486423 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6114db3-979d-47ec-b4a9-4531a031fe79 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:05.486774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6114db3-979d-47ec-b4a9-4531a031fe79 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:05.487019 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6114db3-979d-47ec-b4a9-4531a031fe79 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:05.489970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 208/429] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:51:05.491320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:05.495471 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64405dbd-daa8-4de7-9745-288dc905067e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:05.495772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64405dbd-daa8-4de7-9745-288dc905067e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:05.496065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64405dbd-daa8-4de7-9745-288dc905067e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:05.517953 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61207330-ae17-4b71-8216-d859200a53a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:05.518845 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61207330-ae17-4b71-8216-d859200a53a4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:05.519126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61207330-ae17-4b71-8216-d859200a53a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:05.519399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61207330-ae17-4b71-8216-d859200a53a4 None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:05.521331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61207330-ae17-4b71-8216-d859200a53a4 None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:05.522813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61207330-ae17-4b71-8216-d859200a53a4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:05.524798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 222/430] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:51:05 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:05.530507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61207330-ae17-4b71-8216-d859200a53a4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:05.532086 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-64405dbd-daa8-4de7-9745-288dc905067e None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:05.533794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-64405dbd-daa8-4de7-9745-288dc905067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:05.572915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:05.678246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61207330-ae17-4b71-8216-d859200a53a4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:05.691617 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 209/431] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:05.726341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64405dbd-daa8-4de7-9745-288dc905067e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:05.726822 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9f9fd-25e8-4d36-9d4c-b48ead6d2f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:05.727863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9f9fd-25e8-4d36-9d4c-b48ead6d2f12 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:05.728363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9f9fd-25e8-4d36-9d4c-b48ead6d2f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:05.728804 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9f9fd-25e8-4d36-9d4c-b48ead6d2f12 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:05.730572 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9f9fd-25e8-4d36-9d4c-b48ead6d2f12 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:05.733361 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9f9fd-25e8-4d36-9d4c-b48ead6d2f12 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:05.735545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 223/432] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:51:05 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:05.742481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9f9fd-25e8-4d36-9d4c-b48ead6d2f12 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:05.812530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:05.854688 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9f9fd-25e8-4d36-9d4c-b48ead6d2f12 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:05.859075 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64405dbd-daa8-4de7-9745-288dc905067e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:05.895681 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f81e95-a56c-4be9-8ca7-1a233dfb6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:05.896628 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f81e95-a56c-4be9-8ca7-1a233dfb6a84 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:05.897203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f81e95-a56c-4be9-8ca7-1a233dfb6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:05.897653 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f81e95-a56c-4be9-8ca7-1a233dfb6a84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:05.898641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f81e95-a56c-4be9-8ca7-1a233dfb6a84 None admin] RBAC: Adding query filter params (group.id=c71c537edc9a4df28c3694808669589e, role.id=4498e695105347909981a684f613b6a2, scope.project.id=5589046ca8fb418da62e5cef63cedc9f) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:05.899086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f81e95-a56c-4be9-8ca7-1a233dfb6a84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:05.900779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f81e95-a56c-4be9-8ca7-1a233dfb6a84 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:05.904556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a512f-0f35-46f8-8c4e-69ea874975dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:05.905291 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a512f-0f35-46f8-8c4e-69ea874975dd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:05.905704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a512f-0f35-46f8-8c4e-69ea874975dd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:05.906089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a512f-0f35-46f8-8c4e-69ea874975dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:05.912359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 224/433] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:51:05 2022] GET /identity/v3/role_assignments?group.id=c71c537edc9a4df28c3694808669589e&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=4498e695105347909981a684f613b6a2 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:05.927873 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a512f-0f35-46f8-8c4e-69ea874975dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a22a512f-0f35-46f8-8c4e-69ea874975dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:05.936658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:05.937645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:05.938801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 210/434] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:51:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:05.944966 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f81e95-a56c-4be9-8ca7-1a233dfb6a84 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:06.068438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ef1c8-b68f-4ed0-90ed-7b1a7cc2aa33 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.068680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ef1c8-b68f-4ed0-90ed-7b1a7cc2aa33 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.069212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ef1c8-b68f-4ed0-90ed-7b1a7cc2aa33 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.070920 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 211/435] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:51:06.074905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f81e95-a56c-4be9-8ca7-1a233dfb6a84 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:06.076342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5ffa-e0fb-475d-ad41-50a78edad235 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.076717 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5ffa-e0fb-475d-ad41-50a78edad235 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.077070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5ffa-e0fb-475d-ad41-50a78edad235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.100212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-998f5ffa-e0fb-475d-ad41-50a78edad235 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:06.101616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-998f5ffa-e0fb-475d-ad41-50a78edad235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:06.106427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:06.121913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:06.122665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98745f49-2053-4537-ae2a-dd3184c01d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:06.124101 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98745f49-2053-4537-ae2a-dd3184c01d81 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.124607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98745f49-2053-4537-ae2a-dd3184c01d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.125067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98745f49-2053-4537-ae2a-dd3184c01d81 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.126011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98745f49-2053-4537-ae2a-dd3184c01d81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:06.126505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98745f49-2053-4537-ae2a-dd3184c01d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:06.128199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98745f49-2053-4537-ae2a-dd3184c01d81 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:06.144656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 225/436] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:06.146083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98745f49-2053-4537-ae2a-dd3184c01d81 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:06.200800 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 212/437] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:06.229788 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5ffa-e0fb-475d-ad41-50a78edad235 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:06.277790 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98745f49-2053-4537-ae2a-dd3184c01d81 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:06.329969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bffde-db32-46e4-b325-3177f54bf2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:06.330574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bffde-db32-46e4-b325-3177f54bf2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.330833 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bffde-db32-46e4-b325-3177f54bf2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.331111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bffde-db32-46e4-b325-3177f54bf2c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.370038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5ffa-e0fb-475d-ad41-50a78edad235 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:06.390810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:06.404009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bffde-db32-46e4-b325-3177f54bf2c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:06.405559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bffde-db32-46e4-b325-3177f54bf2c6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:06.408613 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f34cb-14f4-4192-aef8-ea999e5cc760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:06.408613 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f34cb-14f4-4192-aef8-ea999e5cc760 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.408613 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f34cb-14f4-4192-aef8-ea999e5cc760 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.408613 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f34cb-14f4-4192-aef8-ea999e5cc760 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.418916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f34cb-14f4-4192-aef8-ea999e5cc760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ae4f34cb-14f4-4192-aef8-ea999e5cc760 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:06.427285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:06.428353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:06.429279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 213/438] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:51:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:06.429941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0fe6d-9db2-4c37-be00-eef1d3ccdda8 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.430188 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0fe6d-9db2-4c37-be00-eef1d3ccdda8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.430433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0fe6d-9db2-4c37-be00-eef1d3ccdda8 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.431670 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 214/439] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:06.434380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0fe6d-9db2-4c37-be00-eef1d3ccdda8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:06.447089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:06.514141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 226/440] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:06.515423 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030a7ca-4d87-4847-b03d-15f3e3bd770f None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.515710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030a7ca-4d87-4847-b03d-15f3e3bd770f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.515992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030a7ca-4d87-4847-b03d-15f3e3bd770f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.532530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b030a7ca-4d87-4847-b03d-15f3e3bd770f None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:06.533713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b030a7ca-4d87-4847-b03d-15f3e3bd770f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:06.543182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0fe6d-9db2-4c37-be00-eef1d3ccdda8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:06.578273 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a520193-683f-468b-a78b-28e339b5e56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:06.578952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a520193-683f-468b-a78b-28e339b5e56f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.579340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a520193-683f-468b-a78b-28e339b5e56f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.579555 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a520193-683f-468b-a78b-28e339b5e56f None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.590524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a520193-683f-468b-a78b-28e339b5e56f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:06.590524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a520193-683f-468b-a78b-28e339b5e56f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:06.590524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a520193-683f-468b-a78b-28e339b5e56f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:06.591388 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 215/441] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:06.592267 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a520193-683f-468b-a78b-28e339b5e56f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:06.608511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 227/442] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:06.609191 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030a7ca-4d87-4847-b03d-15f3e3bd770f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:06.633974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:06.674090 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:06.705163 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a520193-683f-468b-a78b-28e339b5e56f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:06.713733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030a7ca-4d87-4847-b03d-15f3e3bd770f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:06.740399 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c6aed-93b4-4d1e-9e99-0b9244bb0cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:06.741275 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c6aed-93b4-4d1e-9e99-0b9244bb0cbe None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.741571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c6aed-93b4-4d1e-9e99-0b9244bb0cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.741854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c6aed-93b4-4d1e-9e99-0b9244bb0cbe None admin] PATH_INFO: `/v3/projects/7c4956db4fa4484cbedbb6ce367d55de` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.743078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c6aed-93b4-4d1e-9e99-0b9244bb0cbe None admin] RBAC: Authorizing `identity:get_project(project_id=7c4956db4fa4484cbedbb6ce367d55de)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:06.751221 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c6aed-93b4-4d1e-9e99-0b9244bb0cbe None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:06.752961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 216/443] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:51:06 2022] GET /identity/v3/projects/7c4956db4fa4484cbedbb6ce367d55de => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:06.753149 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439682d-4b21-4583-baef-3f3a61f622dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:06.753768 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c6aed-93b4-4d1e-9e99-0b9244bb0cbe None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:06.753872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439682d-4b21-4583-baef-3f3a61f622dd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.754742 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439682d-4b21-4583-baef-3f3a61f622dd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.754928 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439682d-4b21-4583-baef-3f3a61f622dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.825826 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439682d-4b21-4583-baef-3f3a61f622dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:06.827308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f439682d-4b21-4583-baef-3f3a61f622dd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:06.846443 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:06.856798 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c6aed-93b4-4d1e-9e99-0b9244bb0cbe None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:06.902171 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53733c36-ea34-4886-a656-74b82b264d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:06.903087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53733c36-ea34-4886-a656-74b82b264d18 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.903380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53733c36-ea34-4886-a656-74b82b264d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.903668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53733c36-ea34-4886-a656-74b82b264d18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.914219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:06.919385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53733c36-ea34-4886-a656-74b82b264d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-53733c36-ea34-4886-a656-74b82b264d18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:06.928732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:06.929752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:06.930582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 217/444] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:06.931883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53733c36-ea34-4886-a656-74b82b264d18 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:06.951295 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 228/445] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:06.952426 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439682d-4b21-4583-baef-3f3a61f622dd None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:07.046902 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53733c36-ea34-4886-a656-74b82b264d18 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:07.064852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439682d-4b21-4583-baef-3f3a61f622dd None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:07.084042 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:07.085479 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a86b4-e2be-4dc1-9519-f4a352308f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:07.086351 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a86b4-e2be-4dc1-9519-f4a352308f13 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:07.086644 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a86b4-e2be-4dc1-9519-f4a352308f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:07.087040 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a86b4-e2be-4dc1-9519-f4a352308f13 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:07.088159 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a86b4-e2be-4dc1-9519-f4a352308f13 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:07.089867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a86b4-e2be-4dc1-9519-f4a352308f13 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:07.091328 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 218/446] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:51:06 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:07.092108 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a86b4-e2be-4dc1-9519-f4a352308f13 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:07.107554 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f2532-51e2-4ba3-b80e-f0d1a79d4c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:07.108142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f2532-51e2-4ba3-b80e-f0d1a79d4c5d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:07.108515 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f2532-51e2-4ba3-b80e-f0d1a79d4c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:07.108799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f2532-51e2-4ba3-b80e-f0d1a79d4c5d None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:07.109547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f2532-51e2-4ba3-b80e-f0d1a79d4c5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:07.109850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f2532-51e2-4ba3-b80e-f0d1a79d4c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:07.111274 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f2532-51e2-4ba3-b80e-f0d1a79d4c5d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:07.121091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 229/447] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:07.121912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f2532-51e2-4ba3-b80e-f0d1a79d4c5d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:07.168823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:07.204551 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a86b4-e2be-4dc1-9519-f4a352308f13 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:07.226347 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f2532-51e2-4ba3-b80e-f0d1a79d4c5d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:07.239050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8c434-5681-4821-94ff-c78d60b7b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:07.239911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8c434-5681-4821-94ff-c78d60b7b66a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:07.240195 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8c434-5681-4821-94ff-c78d60b7b66a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:07.240497 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8c434-5681-4821-94ff-c78d60b7b66a None admin] PATH_INFO: `/v3/projects/5589046ca8fb418da62e5cef63cedc9f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:07.241704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8c434-5681-4821-94ff-c78d60b7b66a None admin] RBAC: Authorizing `identity:get_project(project_id=5589046ca8fb418da62e5cef63cedc9f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:07.243159 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8c434-5681-4821-94ff-c78d60b7b66a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:07.244865 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 219/448] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:51:07 2022] GET /identity/v3/projects/5589046ca8fb418da62e5cef63cedc9f => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:07.245628 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8c434-5681-4821-94ff-c78d60b7b66a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:07.266910 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41a9c-27f2-4011-9958-3295eff065f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:07.267913 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41a9c-27f2-4011-9958-3295eff065f5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:07.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41a9c-27f2-4011-9958-3295eff065f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:07.268787 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41a9c-27f2-4011-9958-3295eff065f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:07.270361 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41a9c-27f2-4011-9958-3295eff065f5 None admin] RBAC: Adding query filter params (group.id=c71c537edc9a4df28c3694808669589e, role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=7c4956db4fa4484cbedbb6ce367d55de) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:07.270941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41a9c-27f2-4011-9958-3295eff065f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:07.272650 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41a9c-27f2-4011-9958-3295eff065f5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:07.279091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 230/449] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:51:07 2022] GET /identity/v3/role_assignments?group.id=c71c537edc9a4df28c3694808669589e&scope.project.id=7c4956db4fa4484cbedbb6ce367d55de&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:07.279714 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41a9c-27f2-4011-9958-3295eff065f5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:07.343724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:07.375261 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8c434-5681-4821-94ff-c78d60b7b66a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:07.417010 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41a9c-27f2-4011-9958-3295eff065f5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:07.421953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9e227-835f-4df9-88db-04d884ad9912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:07.423661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9e227-835f-4df9-88db-04d884ad9912 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:07.424490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9e227-835f-4df9-88db-04d884ad9912 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:07.425884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9e227-835f-4df9-88db-04d884ad9912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:07.435145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:07.466449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830bc84-e703-4434-ac51-26f1a2a98980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:07.467328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830bc84-e703-4434-ac51-26f1a2a98980 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:07.467616 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830bc84-e703-4434-ac51-26f1a2a98980 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:07.467900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830bc84-e703-4434-ac51-26f1a2a98980 None admin] PATH_INFO: `/v3/groups/c71c537edc9a4df28c3694808669589e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:07.469565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830bc84-e703-4434-ac51-26f1a2a98980 None admin] RBAC: Authorizing `identity:get_group(group_id=c71c537edc9a4df28c3694808669589e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:07.471173 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830bc84-e703-4434-ac51-26f1a2a98980 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:07.474099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 231/450] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:51:07 2022] GET /identity/v3/groups/c71c537edc9a4df28c3694808669589e => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:07.480496 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830bc84-e703-4434-ac51-26f1a2a98980 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:07.525924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9e227-835f-4df9-88db-04d884ad9912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:07.528489 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9e227-835f-4df9-88db-04d884ad9912 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:07.655722 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830bc84-e703-4434-ac51-26f1a2a98980 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:07.681466 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 220/451] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:07.681911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:07.690078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9e227-835f-4df9-88db-04d884ad9912 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:07.731726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591d9a5-ebc4-4f77-a25b-553cf504c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:07.733276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591d9a5-ebc4-4f77-a25b-553cf504c52a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:07.733574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591d9a5-ebc4-4f77-a25b-553cf504c52a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:07.733882 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591d9a5-ebc4-4f77-a25b-553cf504c52a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:07.734727 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591d9a5-ebc4-4f77-a25b-553cf504c52a None admin] RBAC: Adding query filter params (group.id=c71c537edc9a4df28c3694808669589e, role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=5589046ca8fb418da62e5cef63cedc9f) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:07.735047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591d9a5-ebc4-4f77-a25b-553cf504c52a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:07.736568 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591d9a5-ebc4-4f77-a25b-553cf504c52a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:07.748305 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 232/452] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:51:07 2022] GET /identity/v3/role_assignments?group.id=c71c537edc9a4df28c3694808669589e&scope.project.id=5589046ca8fb418da62e5cef63cedc9f&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 611 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:07.748507 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:07.851647 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9e227-835f-4df9-88db-04d884ad9912 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:07.909660 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69163239-5812-425d-a37d-002118053ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:07.911254 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69163239-5812-425d-a37d-002118053ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:07.911544 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69163239-5812-425d-a37d-002118053ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:07.911827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69163239-5812-425d-a37d-002118053ef9 None admin] PATH_INFO: `/v3/projects/7d718157b3b949fc80592326cc9cfb5a` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:07.913113 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69163239-5812-425d-a37d-002118053ef9 None admin] RBAC: Authorizing `identity:get_project(project_id=7d718157b3b949fc80592326cc9cfb5a)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:07.914573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69163239-5812-425d-a37d-002118053ef9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:07.916319 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 221/453] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:51:07 2022] GET /identity/v3/projects/7d718157b3b949fc80592326cc9cfb5a => generated 344 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:07.922023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591d9a5-ebc4-4f77-a25b-553cf504c52a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:08.051144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591d9a5-ebc4-4f77-a25b-553cf504c52a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:08.086123 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc8021-2369-4ac4-8e94-ebef432b0cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:08.086984 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc8021-2369-4ac4-8e94-ebef432b0cdd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:08.087270 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc8021-2369-4ac4-8e94-ebef432b0cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:08.087556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc8021-2369-4ac4-8e94-ebef432b0cdd None admin] PATH_INFO: `/v3/groups/6a7775b234af40119804bd96800429f9` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:08.088786 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc8021-2369-4ac4-8e94-ebef432b0cdd None admin] RBAC: Authorizing `identity:get_group(group_id=6a7775b234af40119804bd96800429f9)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:08.090252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc8021-2369-4ac4-8e94-ebef432b0cdd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:08.091895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 233/454] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 14:51:07 2022] GET /identity/v3/groups/6a7775b234af40119804bd96800429f9 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:08.098786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69163239-5812-425d-a37d-002118053ef9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:08.206087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69163239-5812-425d-a37d-002118053ef9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:08.208018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:08.242055 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ea073-f3ea-4e45-9945-afa05f0c1325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:08.242716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ea073-f3ea-4e45-9945-afa05f0c1325 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:08.243005 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ea073-f3ea-4e45-9945-afa05f0c1325 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:08.243289 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ea073-f3ea-4e45-9945-afa05f0c1325 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:08.244314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ea073-f3ea-4e45-9945-afa05f0c1325 None admin] RBAC: Adding query filter params (group.id=6a7775b234af40119804bd96800429f9, role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7d718157b3b949fc80592326cc9cfb5a) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:08.244637 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ea073-f3ea-4e45-9945-afa05f0c1325 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:08.246086 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ea073-f3ea-4e45-9945-afa05f0c1325 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:08.251906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 222/455] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 14:51:08 2022] GET /identity/v3/role_assignments?group.id=6a7775b234af40119804bd96800429f9&scope.project.id=7d718157b3b949fc80592326cc9cfb5a&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:10.866505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f27cdd-f86b-4ade-81b0-fcc1c258e1c0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:10.867346 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f27cdd-f86b-4ade-81b0-fcc1c258e1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:10.867346 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f27cdd-f86b-4ade-81b0-fcc1c258e1c0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:10.869116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 234/456] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:10.875874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf47b18-4352-4125-83f0-25ba8ecfa712 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:10.876427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf47b18-4352-4125-83f0-25ba8ecfa712 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:10.876862 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf47b18-4352-4125-83f0-25ba8ecfa712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:10.908596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-aaf47b18-4352-4125-83f0-25ba8ecfa712 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:10.910352 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-aaf47b18-4352-4125-83f0-25ba8ecfa712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:11.023417 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 223/457] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:11.064980 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f27cdd-f86b-4ade-81b0-fcc1c258e1c0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:11.164055 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b0e8e-7f29-476c-b59b-ad418c15823d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.164394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b0e8e-7f29-476c-b59b-ad418c15823d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.164583 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b0e8e-7f29-476c-b59b-ad418c15823d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.164676 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:11.166145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 224/458] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:51:11.172611 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f633ef6-be9b-4851-9711-2e5de8151b4f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.172899 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f633ef6-be9b-4851-9711-2e5de8151b4f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.176329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f633ef6-be9b-4851-9711-2e5de8151b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.213619 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4f633ef6-be9b-4851-9711-2e5de8151b4f None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:11.215857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4f633ef6-be9b-4851-9711-2e5de8151b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:11.263224 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f27cdd-f86b-4ade-81b0-fcc1c258e1c0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:11.326440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 225/459] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:51:11.351368 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f633ef6-be9b-4851-9711-2e5de8151b4f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:11.368992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba800bd2-4659-44b8-9368-519cb85d5630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:11.369953 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba800bd2-4659-44b8-9368-519cb85d5630 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.370112 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba800bd2-4659-44b8-9368-519cb85d5630 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.370403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba800bd2-4659-44b8-9368-519cb85d5630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.384280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:11.397723 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba800bd2-4659-44b8-9368-519cb85d5630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:11.399277 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba800bd2-4659-44b8-9368-519cb85d5630 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ba800bd2-4659-44b8-9368-519cb85d5630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:11.436900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:11.438045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:11.438976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:11.438976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:11.438976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:11.438976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:11.441387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 235/460] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:11.449037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba800bd2-4659-44b8-9368-519cb85d5630 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:11.530037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f633ef6-be9b-4851-9711-2e5de8151b4f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:11.595826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afc56c-6338-49f9-9b03-3e4eea5b037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:11.596701 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afc56c-6338-49f9-9b03-3e4eea5b037a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.597010 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afc56c-6338-49f9-9b03-3e4eea5b037a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.597980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afc56c-6338-49f9-9b03-3e4eea5b037a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.613633 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afc56c-6338-49f9-9b03-3e4eea5b037a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:11.615185 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afc56c-6338-49f9-9b03-3e4eea5b037a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-92afc56c-6338-49f9-9b03-3e4eea5b037a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:11.648885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:11.649826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:11.650721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:11.650721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:11.650721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:11.650721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:11.656442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 226/461] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:11.657229 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:11.658568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e3472-5a8c-49c4-89e1-e35cb710f303 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.660554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e3472-5a8c-49c4-89e1-e35cb710f303 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.661250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e3472-5a8c-49c4-89e1-e35cb710f303 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.664701 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 227/462] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:11.665941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e3472-5a8c-49c4-89e1-e35cb710f303 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:11.668549 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba800bd2-4659-44b8-9368-519cb85d5630 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:11.716455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7e03f-2294-4d06-bcee-fbf500db4fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:11.717063 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7e03f-2294-4d06-bcee-fbf500db4fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.717358 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7e03f-2294-4d06-bcee-fbf500db4fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.717644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7e03f-2294-4d06-bcee-fbf500db4fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.718391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7e03f-2294-4d06-bcee-fbf500db4fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:11.718706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7e03f-2294-4d06-bcee-fbf500db4fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:11.720281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7e03f-2294-4d06-bcee-fbf500db4fb6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:11.741384 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 236/463] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:11.746137 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b901c80-4276-44b6-9f54-8839853f41f6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.746137 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b901c80-4276-44b6-9f54-8839853f41f6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.746137 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b901c80-4276-44b6-9f54-8839853f41f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.762149 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7b901c80-4276-44b6-9f54-8839853f41f6 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:11.764001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7b901c80-4276-44b6-9f54-8839853f41f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:11.771707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:11.806337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e3472-5a8c-49c4-89e1-e35cb710f303 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:11.845742 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f6d1a-c4aa-47a9-b9c5-4e9c7378dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:11.846684 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6d1a-c4aa-47a9-b9c5-4e9c7378dc0d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.846973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6d1a-c4aa-47a9-b9c5-4e9c7378dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.847360 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f6d1a-c4aa-47a9-b9c5-4e9c7378dc0d None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.848130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6d1a-c4aa-47a9-b9c5-4e9c7378dc0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:11.848442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6d1a-c4aa-47a9-b9c5-4e9c7378dc0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:11.849833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f6d1a-c4aa-47a9-b9c5-4e9c7378dc0d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:11.857774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 237/464] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:11.859147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7367de-96da-4652-9758-c0f68276fb05 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.859442 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7367de-96da-4652-9758-c0f68276fb05 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.859753 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7367de-96da-4652-9758-c0f68276fb05 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.867341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 238/465] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:11.867341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7367de-96da-4652-9758-c0f68276fb05 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:11.869505 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 228/466] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:11.870960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238868e9-492b-4fd4-bae6-46b30e7be937 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.871461 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238868e9-492b-4fd4-bae6-46b30e7be937 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.871857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238868e9-492b-4fd4-bae6-46b30e7be937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.897827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-238868e9-492b-4fd4-bae6-46b30e7be937 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:11.899508 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-238868e9-492b-4fd4-bae6-46b30e7be937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:11.982549 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:11.996260 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 229/467] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:11.996260 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238868e9-492b-4fd4-bae6-46b30e7be937 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:12.011241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7367de-96da-4652-9758-c0f68276fb05 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:12.066706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e086c-285f-4b67-8ea7-2516a51c33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:12.067467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e086c-285f-4b67-8ea7-2516a51c33cb None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:12.067761 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e086c-285f-4b67-8ea7-2516a51c33cb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:12.068046 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e086c-285f-4b67-8ea7-2516a51c33cb None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:12.068819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e086c-285f-4b67-8ea7-2516a51c33cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:12.070344 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e086c-285f-4b67-8ea7-2516a51c33cb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:12.087181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:12.124109 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238868e9-492b-4fd4-bae6-46b30e7be937 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:12.162373 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a062f36-aada-438c-a4ec-09b2dc023d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:12.163049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a062f36-aada-438c-a4ec-09b2dc023d01 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:12.163341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a062f36-aada-438c-a4ec-09b2dc023d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:12.163622 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a062f36-aada-438c-a4ec-09b2dc023d01 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:12.164353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a062f36-aada-438c-a4ec-09b2dc023d01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:12.165858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a062f36-aada-438c-a4ec-09b2dc023d01 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:12.228432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 239/468] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:51:11 2022] POST /identity/v3/users => generated 246 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:12.229544 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e086c-285f-4b67-8ea7-2516a51c33cb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:12.271937 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 230/469] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:51:11 2022] POST /identity/v3/users => generated 248 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:12.273837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a062f36-aada-438c-a4ec-09b2dc023d01 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:12.459846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e086c-285f-4b67-8ea7-2516a51c33cb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:12.502049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a062f36-aada-438c-a4ec-09b2dc023d01 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:12.544415 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bc12c-dde9-4b51-9327-99777060db94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:12.548093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bc12c-dde9-4b51-9327-99777060db94 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:12.548093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bc12c-dde9-4b51-9327-99777060db94 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:12.548093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bc12c-dde9-4b51-9327-99777060db94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:12.556494 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:12.569443 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bc12c-dde9-4b51-9327-99777060db94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:12.572133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6bc12c-dde9-4b51-9327-99777060db94 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:12.577085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:12.588353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0b269-e0b3-46c3-be95-5a3c810bd001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:12.588353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0b269-e0b3-46c3-be95-5a3c810bd001 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:12.588353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0b269-e0b3-46c3-be95-5a3c810bd001 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:12.588353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0b269-e0b3-46c3-be95-5a3c810bd001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ae6bc12c-dde9-4b51-9327-99777060db94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:12.602684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:12.603860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:12.605079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:12.605079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:12.605079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:12.605079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:12.605079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 240/470] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:12.610131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bc12c-dde9-4b51-9327-99777060db94 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:12.611639 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0b269-e0b3-46c3-be95-5a3c810bd001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:12.613921 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0b269-e0b3-46c3-be95-5a3c810bd001 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c6b0b269-e0b3-46c3-be95-5a3c810bd001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:12.636924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:12.637945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:12.638922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:12.638922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:12.638922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:12.638922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:12.640967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 231/471] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:12.646339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0b269-e0b3-46c3-be95-5a3c810bd001 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:12.812671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bc12c-dde9-4b51-9327-99777060db94 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:12.821716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0b269-e0b3-46c3-be95-5a3c810bd001 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:12.862513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c31913-778d-4b08-80ed-fb992fd9d9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:12.863919 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c31913-778d-4b08-80ed-fb992fd9d9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:12.864314 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c31913-778d-4b08-80ed-fb992fd9d9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:12.864608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c31913-778d-4b08-80ed-fb992fd9d9d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:12.864874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00279d4-6c2f-4271-8dd3-0217c136e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:12.865461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c31913-778d-4b08-80ed-fb992fd9d9d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:12.865570 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00279d4-6c2f-4271-8dd3-0217c136e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:12.865781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c31913-778d-4b08-80ed-fb992fd9d9d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:12.865883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00279d4-6c2f-4271-8dd3-0217c136e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:12.866092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00279d4-6c2f-4271-8dd3-0217c136e8e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:12.866895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00279d4-6c2f-4271-8dd3-0217c136e8e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:12.867212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00279d4-6c2f-4271-8dd3-0217c136e8e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:12.867314 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c31913-778d-4b08-80ed-fb992fd9d9d7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:12.868958 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00279d4-6c2f-4271-8dd3-0217c136e8e4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:12.880057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 241/472] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:12.881601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34623526-784f-4495-aa63-5e96895d88a0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:12.881896 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34623526-784f-4495-aa63-5e96895d88a0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:12.882177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34623526-784f-4495-aa63-5e96895d88a0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:12.885225 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 232/474] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:12.886942 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00279d4-6c2f-4271-8dd3-0217c136e8e4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:12.887886 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 242/474] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:12.889852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf42b-b6a7-461e-a5ac-3a5bac3b7c1e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:12.890151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf42b-b6a7-461e-a5ac-3a5bac3b7c1e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:12.890439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf42b-b6a7-461e-a5ac-3a5bac3b7c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:12.907558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2ecbf42b-b6a7-461e-a5ac-3a5bac3b7c1e None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:12.908801 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2ecbf42b-b6a7-461e-a5ac-3a5bac3b7c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:12.946387 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:12.973091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:13.004967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 243/475] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:51:13.006437 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf42b-b6a7-461e-a5ac-3a5bac3b7c1e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:13.038707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00279d4-6c2f-4271-8dd3-0217c136e8e4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:13.082610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8a4b8-5477-4062-9638-77776f09b479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:13.083406 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8a4b8-5477-4062-9638-77776f09b479 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:13.083702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8a4b8-5477-4062-9638-77776f09b479 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:13.083988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8a4b8-5477-4062-9638-77776f09b479 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:13.084754 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8a4b8-5477-4062-9638-77776f09b479 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:13.086762 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8a4b8-5477-4062-9638-77776f09b479 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:13.154713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf42b-b6a7-461e-a5ac-3a5bac3b7c1e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:13.191338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e98c87-8cf7-4925-98ca-2418ef3b2587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:13.192018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e98c87-8cf7-4925-98ca-2418ef3b2587 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:13.192419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e98c87-8cf7-4925-98ca-2418ef3b2587 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:13.192707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e98c87-8cf7-4925-98ca-2418ef3b2587 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:13.193416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e98c87-8cf7-4925-98ca-2418ef3b2587 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:13.194999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e98c87-8cf7-4925-98ca-2418ef3b2587 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:13.206096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 233/476] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:51:12 2022] POST /identity/v3/users => generated 248 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:13.206096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8a4b8-5477-4062-9638-77776f09b479 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:13.307667 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 244/477] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:51:13 2022] POST /identity/v3/users => generated 249 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:13.351466 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:13.364419 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8a4b8-5477-4062-9638-77776f09b479 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:13.410615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ff385-570f-4693-883e-58ae158489aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:13.411355 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ff385-570f-4693-883e-58ae158489aa None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:13.411645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ff385-570f-4693-883e-58ae158489aa None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:13.411928 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ff385-570f-4693-883e-58ae158489aa None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:13.412741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ff385-570f-4693-883e-58ae158489aa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:13.414162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ff385-570f-4693-883e-58ae158489aa None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:13.460783 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.sql.core [None req-028ff385-570f-4693-883e-58ae158489aa None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 14:51:13.460783 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 14:51:13.460783 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [parameters: {'id': '41d96c8e38ab4c639eddff7b2bbd78f5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 14:51:13.460783 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80420) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-028ff385-570f-4693-883e-58ae158489aa None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result.read() Jun 16 14:51:13.477782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:51:13.478769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application yield resource Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise self.value Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application session.commit() Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 14:51:13.479918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.session.flush() Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application compat.raise_( Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 14:51:13.480802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:51:13.481905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result.read() Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:51:13.482763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [parameters: {'id': '41d96c8e38ab4c639eddff7b2bbd78f5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:13.486868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 14:51:13.489826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:13.491905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 234/478] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 14:51:13 2022] POST /identity/v3/roles => generated 144 bytes in 275 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:13.492061 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e98c87-8cf7-4925-98ca-2418ef3b2587 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:13.548369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:13.730989 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e98c87-8cf7-4925-98ca-2418ef3b2587 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:13.785213 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801fa2e-55ad-47c5-b2ba-bcd609b570db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:13.786082 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801fa2e-55ad-47c5-b2ba-bcd609b570db None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:13.786374 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801fa2e-55ad-47c5-b2ba-bcd609b570db None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:13.786656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801fa2e-55ad-47c5-b2ba-bcd609b570db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:13.802957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801fa2e-55ad-47c5-b2ba-bcd609b570db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-7801fa2e-55ad-47c5-b2ba-bcd609b570db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:13.815694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:13.816778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:13.817766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 245/479] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:51:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:13.823057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ff385-570f-4693-883e-58ae158489aa None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:13.967654 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ff385-570f-4693-883e-58ae158489aa None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:14.018445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4f5ac-55cb-41c5-9c5c-04c892e437cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:14.019595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4f5ac-55cb-41c5-9c5c-04c892e437cf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:14.019886 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4f5ac-55cb-41c5-9c5c-04c892e437cf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:14.020174 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4f5ac-55cb-41c5-9c5c-04c892e437cf None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:14.021093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4f5ac-55cb-41c5-9c5c-04c892e437cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:14.021410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4f5ac-55cb-41c5-9c5c-04c892e437cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:14.022807 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4f5ac-55cb-41c5-9c5c-04c892e437cf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:14.036151 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 235/480] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:15.011291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d6e9-1833-4837-9bee-9ae47e072b13 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:15.012375 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d6e9-1833-4837-9bee-9ae47e072b13 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:15.012876 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d6e9-1833-4837-9bee-9ae47e072b13 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:15.015159 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 246/481] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:15.022803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49809e4d-1123-49ad-8445-0195d04dcad2 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:15.023975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49809e4d-1123-49ad-8445-0195d04dcad2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:15.024632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49809e4d-1123-49ad-8445-0195d04dcad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:15.054731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-49809e4d-1123-49ad-8445-0195d04dcad2 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:15.056296 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-49809e4d-1123-49ad-8445-0195d04dcad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:15.083294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:15.158541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 236/482] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:15.187398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870ddc0-ddff-488f-bee8-268da3dd9497 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:15.187699 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870ddc0-ddff-488f-bee8-268da3dd9497 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:15.187971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870ddc0-ddff-488f-bee8-268da3dd9497 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:15.189415 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 247/483] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:15.195851 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3075d0-a5e9-4ef4-b527-4e38df5260c4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:15.196146 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3075d0-a5e9-4ef4-b527-4e38df5260c4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:15.196487 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3075d0-a5e9-4ef4-b527-4e38df5260c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:15.198752 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870ddc0-ddff-488f-bee8-268da3dd9497 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:15.218405 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7a3075d0-a5e9-4ef4-b527-4e38df5260c4 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:15.219602 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7a3075d0-a5e9-4ef4-b527-4e38df5260c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:15.346725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 237/484] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:15.377669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:15.382334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870ddc0-ddff-488f-bee8-268da3dd9497 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:15.413741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3075d0-a5e9-4ef4-b527-4e38df5260c4 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:15.425445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdd419-e670-4937-a2f3-6f74fb0651d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:15.426331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdd419-e670-4937-a2f3-6f74fb0651d4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:15.426624 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdd419-e670-4937-a2f3-6f74fb0651d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:15.426966 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdd419-e670-4937-a2f3-6f74fb0651d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:15.444716 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afdd419-e670-4937-a2f3-6f74fb0651d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9afdd419-e670-4937-a2f3-6f74fb0651d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:15.452694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:15.453674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:15.454558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 248/485] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:15.460024 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdd419-e670-4937-a2f3-6f74fb0651d4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:15.537833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:15.598385 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdd419-e670-4937-a2f3-6f74fb0651d4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:15.631088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3075d0-a5e9-4ef4-b527-4e38df5260c4 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:15.637863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878f769-e680-4f7f-8956-11929f354805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:15.638739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878f769-e680-4f7f-8956-11929f354805 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:15.639204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878f769-e680-4f7f-8956-11929f354805 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:15.639634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878f769-e680-4f7f-8956-11929f354805 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:15.640698 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878f769-e680-4f7f-8956-11929f354805 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:15.641419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878f769-e680-4f7f-8956-11929f354805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:15.643198 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878f769-e680-4f7f-8956-11929f354805 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:15.653366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 249/486] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:15.658956 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878f769-e680-4f7f-8956-11929f354805 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:15.685968 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:15.713701 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6068cd8-e5f7-42aa-8eed-a4a533748af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:15.714522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068cd8-e5f7-42aa-8eed-a4a533748af5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:15.714855 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068cd8-e5f7-42aa-8eed-a4a533748af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:15.715136 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068cd8-e5f7-42aa-8eed-a4a533748af5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:15.744872 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6068cd8-e5f7-42aa-8eed-a4a533748af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c6068cd8-e5f7-42aa-8eed-a4a533748af5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:15.752993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:15.754345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:15.755364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 238/487] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:15.760424 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6068cd8-e5f7-42aa-8eed-a4a533748af5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:15.774668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878f769-e680-4f7f-8956-11929f354805 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:15.815558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8203467-030f-40da-8adb-2bcbb3b3022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:15.817199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8203467-030f-40da-8adb-2bcbb3b3022e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:15.817662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8203467-030f-40da-8adb-2bcbb3b3022e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:15.818092 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8203467-030f-40da-8adb-2bcbb3b3022e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:15.831674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8203467-030f-40da-8adb-2bcbb3b3022e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:15.833565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8203467-030f-40da-8adb-2bcbb3b3022e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-b8203467-030f-40da-8adb-2bcbb3b3022e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:15.853987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:15.855091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:15.856206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:15.856206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:15.856206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:15.856206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:15.858143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 250/488] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:15.866455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8203467-030f-40da-8adb-2bcbb3b3022e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:15.967668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6068cd8-e5f7-42aa-8eed-a4a533748af5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:15.971473 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:15.999272 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8203467-030f-40da-8adb-2bcbb3b3022e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:16.021165 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31126c-a141-4fd8-b474-5ba250e69637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.022171 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31126c-a141-4fd8-b474-5ba250e69637 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.022597 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31126c-a141-4fd8-b474-5ba250e69637 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.023018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31126c-a141-4fd8-b474-5ba250e69637 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.024496 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31126c-a141-4fd8-b474-5ba250e69637 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:16.025039 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31126c-a141-4fd8-b474-5ba250e69637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:16.026547 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31126c-a141-4fd8-b474-5ba250e69637 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:16.033098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:16.036680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369724c-5b44-46ea-9c01-391358351a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.037446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369724c-5b44-46ea-9c01-391358351a02 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.037713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369724c-5b44-46ea-9c01-391358351a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.038008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369724c-5b44-46ea-9c01-391358351a02 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.041382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369724c-5b44-46ea-9c01-391358351a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:16.041382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369724c-5b44-46ea-9c01-391358351a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:16.041382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369724c-5b44-46ea-9c01-391358351a02 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:16.044352 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 239/489] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:16.045941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb351-ceea-4035-9d19-5b1a98349413 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.046233 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb351-ceea-4035-9d19-5b1a98349413 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.046517 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb351-ceea-4035-9d19-5b1a98349413 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.052404 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 240/490] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:16.053554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fb351-ceea-4035-9d19-5b1a98349413 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:16.055696 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 251/491] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:16.061905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b03ec0-d656-4db1-844f-0f48b4b223f9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.062379 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b03ec0-d656-4db1-844f-0f48b4b223f9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.062813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b03ec0-d656-4db1-844f-0f48b4b223f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.084536 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-58b03ec0-d656-4db1-844f-0f48b4b223f9 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:16.086608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-58b03ec0-d656-4db1-844f-0f48b4b223f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:16.177740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 252/492] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:16.178814 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b03ec0-d656-4db1-844f-0f48b4b223f9 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:16.187852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fb351-ceea-4035-9d19-5b1a98349413 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:16.246643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b0aeb-fe8b-4444-baae-982ee3a42ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.248461 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b0aeb-fe8b-4444-baae-982ee3a42ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.249250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b0aeb-fe8b-4444-baae-982ee3a42ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.250010 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b0aeb-fe8b-4444-baae-982ee3a42ad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.258518 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:16.267552 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b0aeb-fe8b-4444-baae-982ee3a42ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:16.270181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b0aeb-fe8b-4444-baae-982ee3a42ad0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-294b0aeb-fe8b-4444-baae-982ee3a42ad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:16.293515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:16.294630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:16.295700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:16.295700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:16.295700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:16.295700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:16.302708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 241/493] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:16.304313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b0aeb-fe8b-4444-baae-982ee3a42ad0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:16.306598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b03ec0-d656-4db1-844f-0f48b4b223f9 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:16.345749 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e83d5c-f07e-439e-9c9c-bd297c80dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.346556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e83d5c-f07e-439e-9c9c-bd297c80dd9b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.346835 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e83d5c-f07e-439e-9c9c-bd297c80dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.347835 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e83d5c-f07e-439e-9c9c-bd297c80dd9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.355578 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e83d5c-f07e-439e-9c9c-bd297c80dd9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:16.357129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e83d5c-f07e-439e-9c9c-bd297c80dd9b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-49e83d5c-f07e-439e-9c9c-bd297c80dd9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:16.374011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:16.375001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:16.375990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:16.377025 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:16.377099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 253/494] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 14:51:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:16.377099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e83d5c-f07e-439e-9c9c-bd297c80dd9b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:16.436805 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b0aeb-fe8b-4444-baae-982ee3a42ad0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:16.476682 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571adf5d-894d-4f26-a88f-60ef31718074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.477608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571adf5d-894d-4f26-a88f-60ef31718074 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.477869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571adf5d-894d-4f26-a88f-60ef31718074 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.478149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571adf5d-894d-4f26-a88f-60ef31718074 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.490106 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571adf5d-894d-4f26-a88f-60ef31718074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-571adf5d-894d-4f26-a88f-60ef31718074 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:16.497909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:16.499038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:16.499904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:16.506077 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 242/495] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:16.506077 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df973d-7961-4e4f-b889-65c0a1b4b78d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.506077 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df973d-7961-4e4f-b889-65c0a1b4b78d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.506077 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df973d-7961-4e4f-b889-65c0a1b4b78d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.506437 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e83d5c-f07e-439e-9c9c-bd297c80dd9b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:16.506512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 243/496] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:16.508017 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df973d-7961-4e4f-b889-65c0a1b4b78d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:16.544827 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:16.559055 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6836f3-08c2-4818-a054-3887cb1b4c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.560538 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6836f3-08c2-4818-a054-3887cb1b4c9b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.561004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6836f3-08c2-4818-a054-3887cb1b4c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.561951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6836f3-08c2-4818-a054-3887cb1b4c9b None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.563823 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6836f3-08c2-4818-a054-3887cb1b4c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:16.564603 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6836f3-08c2-4818-a054-3887cb1b4c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:16.568181 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6836f3-08c2-4818-a054-3887cb1b4c9b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:16.585667 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 254/497] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:16.586689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6836f3-08c2-4818-a054-3887cb1b4c9b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:16.636571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df973d-7961-4e4f-b889-65c0a1b4b78d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:16.651222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:16.676192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86442ee5-6c11-4bc5-bbd4-6e1768a4df18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.677020 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86442ee5-6c11-4bc5-bbd4-6e1768a4df18 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.677306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86442ee5-6c11-4bc5-bbd4-6e1768a4df18 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.677585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86442ee5-6c11-4bc5-bbd4-6e1768a4df18 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.678455 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86442ee5-6c11-4bc5-bbd4-6e1768a4df18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:16.678767 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86442ee5-6c11-4bc5-bbd4-6e1768a4df18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:16.680218 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86442ee5-6c11-4bc5-bbd4-6e1768a4df18 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:16.709351 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6836f3-08c2-4818-a054-3887cb1b4c9b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:16.730922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 244/498] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 14:51:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:16.730922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619a857-846b-46b7-b22c-75e66c8cb1b9 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.730922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619a857-846b-46b7-b22c-75e66c8cb1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.730922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619a857-846b-46b7-b22c-75e66c8cb1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.746808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72341a4c-5c62-4c62-ad35-f81d4f5a7914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.747485 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72341a4c-5c62-4c62-ad35-f81d4f5a7914 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.747799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72341a4c-5c62-4c62-ad35-f81d4f5a7914 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.748083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72341a4c-5c62-4c62-ad35-f81d4f5a7914 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.749102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72341a4c-5c62-4c62-ad35-f81d4f5a7914 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:16.749473 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e619a857-846b-46b7-b22c-75e66c8cb1b9 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:16.749573 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72341a4c-5c62-4c62-ad35-f81d4f5a7914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:16.750478 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e619a857-846b-46b7-b22c-75e66c8cb1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:16.750960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72341a4c-5c62-4c62-ad35-f81d4f5a7914 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:16.760127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 255/499] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:16.760817 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72341a4c-5c62-4c62-ad35-f81d4f5a7914 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:16.814524 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:16.844465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 245/500] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:16.844465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619a857-846b-46b7-b22c-75e66c8cb1b9 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:16.884463 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72341a4c-5c62-4c62-ad35-f81d4f5a7914 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:16.919141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684ff9a-139b-49dd-bdfe-bfe05529fb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.919866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684ff9a-139b-49dd-bdfe-bfe05529fb05 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.920157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684ff9a-139b-49dd-bdfe-bfe05529fb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.920566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684ff9a-139b-49dd-bdfe-bfe05529fb05 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:16.928745 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684ff9a-139b-49dd-bdfe-bfe05529fb05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:16.930210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684ff9a-139b-49dd-bdfe-bfe05529fb05 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-8684ff9a-139b-49dd-bdfe-bfe05529fb05 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:16.944482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:16.945432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:16.946336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 256/501] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:16.946896 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:16.946967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684ff9a-139b-49dd-bdfe-bfe05529fb05 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:16.950097 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619a857-846b-46b7-b22c-75e66c8cb1b9 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:16.985816 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647d0fd-45c7-41d6-bfda-c4fa4b6948f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:16.986384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647d0fd-45c7-41d6-bfda-c4fa4b6948f0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:16.986670 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647d0fd-45c7-41d6-bfda-c4fa4b6948f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:16.986949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647d0fd-45c7-41d6-bfda-c4fa4b6948f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.050872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684ff9a-139b-49dd-bdfe-bfe05529fb05 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.055954 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647d0fd-45c7-41d6-bfda-c4fa4b6948f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:17.057461 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647d0fd-45c7-41d6-bfda-c4fa4b6948f0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:17.058502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:17.086934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831140ec-6767-47de-ae73-e0970a387cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:17.087732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831140ec-6767-47de-ae73-e0970a387cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.088026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831140ec-6767-47de-ae73-e0970a387cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.088341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831140ec-6767-47de-ae73-e0970a387cf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.099451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831140ec-6767-47de-ae73-e0970a387cf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:17.100952 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831140ec-6767-47de-ae73-e0970a387cf4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-831140ec-6767-47de-ae73-e0970a387cf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:17.118285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:17.119394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:17.120288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:17.120288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:17.120288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:17.120288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:17.120288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 257/502] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:17.120288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831140ec-6767-47de-ae73-e0970a387cf4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:17.159876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:17.163489 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 246/503] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:17.164845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a183f3-d0d1-4319-a723-49fb812fb901 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.165138 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a183f3-d0d1-4319-a723-49fb812fb901 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.165428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a183f3-d0d1-4319-a723-49fb812fb901 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.166837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 247/504] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:17.167458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a183f3-d0d1-4319-a723-49fb812fb901 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:17.227157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831140ec-6767-47de-ae73-e0970a387cf4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.261687 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67470621-721a-47dd-b7d8-6b82721fafad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:17.262401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67470621-721a-47dd-b7d8-6b82721fafad None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.262689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67470621-721a-47dd-b7d8-6b82721fafad None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.262972 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67470621-721a-47dd-b7d8-6b82721fafad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.270252 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a183f3-d0d1-4319-a723-49fb812fb901 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.273618 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67470621-721a-47dd-b7d8-6b82721fafad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-67470621-721a-47dd-b7d8-6b82721fafad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:17.281251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:17.282185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:17.282986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 258/505] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:17.283642 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67470621-721a-47dd-b7d8-6b82721fafad None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:17.304024 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:17.308926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11260587-8516-488b-97fd-5238428c7f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:17.309606 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11260587-8516-488b-97fd-5238428c7f6e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.309897 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11260587-8516-488b-97fd-5238428c7f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.310188 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11260587-8516-488b-97fd-5238428c7f6e None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.311104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11260587-8516-488b-97fd-5238428c7f6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:17.311437 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11260587-8516-488b-97fd-5238428c7f6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:17.312922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11260587-8516-488b-97fd-5238428c7f6e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:17.332138 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 248/506] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:17.333003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11260587-8516-488b-97fd-5238428c7f6e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:17.387359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67470621-721a-47dd-b7d8-6b82721fafad None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.392451 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:17.421256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b292156-448a-43a1-80e8-511a7d110cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:17.421915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b292156-448a-43a1-80e8-511a7d110cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.422202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b292156-448a-43a1-80e8-511a7d110cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.422484 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b292156-448a-43a1-80e8-511a7d110cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.423210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b292156-448a-43a1-80e8-511a7d110cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:17.423519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b292156-448a-43a1-80e8-511a7d110cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:17.425002 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b292156-448a-43a1-80e8-511a7d110cd9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:17.434946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11260587-8516-488b-97fd-5238428c7f6e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.436707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 259/507] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:17.437914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85b758-103d-42eb-9776-7b45f8fcaee9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.438203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85b758-103d-42eb-9776-7b45f8fcaee9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.438484 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85b758-103d-42eb-9776-7b45f8fcaee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.454671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8d85b758-103d-42eb-9776-7b45f8fcaee9 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:17.455862 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8d85b758-103d-42eb-9776-7b45f8fcaee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:17.470832 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a093010-a7a6-4d9a-93e1-fc12e83b3f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:17.471629 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a093010-a7a6-4d9a-93e1-fc12e83b3f8c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.472019 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a093010-a7a6-4d9a-93e1-fc12e83b3f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.472502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a093010-a7a6-4d9a-93e1-fc12e83b3f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.482702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a093010-a7a6-4d9a-93e1-fc12e83b3f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:17.484188 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a093010-a7a6-4d9a-93e1-fc12e83b3f8c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4a093010-a7a6-4d9a-93e1-fc12e83b3f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:17.502224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:17.504695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:17.505867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:17.505867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:17.505867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:17.505867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:17.505867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 249/508] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:17.505867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a093010-a7a6-4d9a-93e1-fc12e83b3f8c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:17.527151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 260/509] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:17.528155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85b758-103d-42eb-9776-7b45f8fcaee9 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:17.548313 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:17.611947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a093010-a7a6-4d9a-93e1-fc12e83b3f8c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.622688 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:17.640850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85b758-103d-42eb-9776-7b45f8fcaee9 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.647296 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eadb9-0efc-40dd-bf62-6ee8d1627c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:17.647975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eadb9-0efc-40dd-bf62-6ee8d1627c15 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.648291 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eadb9-0efc-40dd-bf62-6ee8d1627c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.648572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eadb9-0efc-40dd-bf62-6ee8d1627c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.649388 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eadb9-0efc-40dd-bf62-6ee8d1627c15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:17.649695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eadb9-0efc-40dd-bf62-6ee8d1627c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:17.651070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eadb9-0efc-40dd-bf62-6ee8d1627c15 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:17.664944 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 250/510] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:17.664944 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eadb9-0efc-40dd-bf62-6ee8d1627c15 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:17.677854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528fac2-1c53-483d-979d-2f4dd6f670b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:17.678506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528fac2-1c53-483d-979d-2f4dd6f670b9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.678793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528fac2-1c53-483d-979d-2f4dd6f670b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.679073 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528fac2-1c53-483d-979d-2f4dd6f670b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.756618 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528fac2-1c53-483d-979d-2f4dd6f670b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:17.758356 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528fac2-1c53-483d-979d-2f4dd6f670b9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:17.764200 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eadb9-0efc-40dd-bf62-6ee8d1627c15 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.767882 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:17.800530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121e001-8292-4f2f-982e-fc5b9d7b48b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:17.801544 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121e001-8292-4f2f-982e-fc5b9d7b48b0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.801833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121e001-8292-4f2f-982e-fc5b9d7b48b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.802113 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121e001-8292-4f2f-982e-fc5b9d7b48b0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.809550 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121e001-8292-4f2f-982e-fc5b9d7b48b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:17.810947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121e001-8292-4f2f-982e-fc5b9d7b48b0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5121e001-8292-4f2f-982e-fc5b9d7b48b0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:17.827635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:17.828949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 251/511] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:17.830006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121e001-8292-4f2f-982e-fc5b9d7b48b0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:17.863546 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:17.866266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 261/512] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:17.867104 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528fac2-1c53-483d-979d-2f4dd6f670b9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:17.943153 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121e001-8292-4f2f-982e-fc5b9d7b48b0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.972471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528fac2-1c53-483d-979d-2f4dd6f670b9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:17.981797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c06207-60e2-47d8-bee6-34d1e961f0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:17.982008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:17.982458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c06207-60e2-47d8-bee6-34d1e961f0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:17.982752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c06207-60e2-47d8-bee6-34d1e961f0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:17.983035 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c06207-60e2-47d8-bee6-34d1e961f0c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:17.983770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c06207-60e2-47d8-bee6-34d1e961f0c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:17.984085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c06207-60e2-47d8-bee6-34d1e961f0c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:17.985508 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c06207-60e2-47d8-bee6-34d1e961f0c8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:17.997670 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 252/513] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:17.998504 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c06207-60e2-47d8-bee6-34d1e961f0c8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.010019 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81db77-141d-476c-9c90-a4464d3890f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.011176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81db77-141d-476c-9c90-a4464d3890f6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.011471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81db77-141d-476c-9c90-a4464d3890f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.011753 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81db77-141d-476c-9c90-a4464d3890f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.023268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81db77-141d-476c-9c90-a4464d3890f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.024722 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81db77-141d-476c-9c90-a4464d3890f6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-af81db77-141d-476c-9c90-a4464d3890f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:18.044192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:18.045116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:18.045969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:18.045969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:18.045969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:18.045969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:18.045969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 262/514] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:18.046228 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81db77-141d-476c-9c90-a4464d3890f6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.097769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:18.107127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c06207-60e2-47d8-bee6-34d1e961f0c8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.143299 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950da60-810a-47ba-81bd-b4ef5e92c7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.145070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950da60-810a-47ba-81bd-b4ef5e92c7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.145070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950da60-810a-47ba-81bd-b4ef5e92c7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.145326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950da60-810a-47ba-81bd-b4ef5e92c7e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.155470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950da60-810a-47ba-81bd-b4ef5e92c7e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.156954 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950da60-810a-47ba-81bd-b4ef5e92c7e6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.157567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81db77-141d-476c-9c90-a4464d3890f6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-8950da60-810a-47ba-81bd-b4ef5e92c7e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:18.176572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:18.178037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:18.179279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:18.179279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:18.179279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:18.179279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:18.179279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 253/515] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:18.179279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950da60-810a-47ba-81bd-b4ef5e92c7e6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.201973 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba620a7b-33b9-45c3-9dab-ec7b1b18be37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.202637 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba620a7b-33b9-45c3-9dab-ec7b1b18be37 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.202922 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba620a7b-33b9-45c3-9dab-ec7b1b18be37 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.203246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba620a7b-33b9-45c3-9dab-ec7b1b18be37 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.204110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba620a7b-33b9-45c3-9dab-ec7b1b18be37 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:18.204471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba620a7b-33b9-45c3-9dab-ec7b1b18be37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.205933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba620a7b-33b9-45c3-9dab-ec7b1b18be37 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.244619 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 263/516] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:18.245649 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba620a7b-33b9-45c3-9dab-ec7b1b18be37 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.267432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:18.293960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950da60-810a-47ba-81bd-b4ef5e92c7e6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.329309 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2565-6ed9-4822-95aa-7afef6873cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.330108 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2565-6ed9-4822-95aa-7afef6873cba None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.330399 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2565-6ed9-4822-95aa-7afef6873cba None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.330674 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2565-6ed9-4822-95aa-7afef6873cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.341223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad2565-6ed9-4822-95aa-7afef6873cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.342617 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad2565-6ed9-4822-95aa-7afef6873cba None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.345245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:18.352724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba620a7b-33b9-45c3-9dab-ec7b1b18be37 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-67ad2565-6ed9-4822-95aa-7afef6873cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:18.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:18.362253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:18.363936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:18.363936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:18.363936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:18.363936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:18.363936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 254/517] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:18.363936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2565-6ed9-4822-95aa-7afef6873cba None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.391123 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578ad6c-6439-4d87-91e6-d22b6854a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.391911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578ad6c-6439-4d87-91e6-d22b6854a38a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.392196 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578ad6c-6439-4d87-91e6-d22b6854a38a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.392502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578ad6c-6439-4d87-91e6-d22b6854a38a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.400372 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578ad6c-6439-4d87-91e6-d22b6854a38a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.401858 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578ad6c-6439-4d87-91e6-d22b6854a38a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-4578ad6c-6439-4d87-91e6-d22b6854a38a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:18.417067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:18.418107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:18.418999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:18.418999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:18.418999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:18.418999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:18.418999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:18.418999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:18.418999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:18.418999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:18.418999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 264/518] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:18.419843 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578ad6c-6439-4d87-91e6-d22b6854a38a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.469286 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2565-6ed9-4822-95aa-7afef6873cba None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.505760 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:18.506703 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bce64-87e3-465c-b8ca-fc63f1d0f93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.507390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bce64-87e3-465c-b8ca-fc63f1d0f93a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.507933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bce64-87e3-465c-b8ca-fc63f1d0f93a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.508217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bce64-87e3-465c-b8ca-fc63f1d0f93a None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.508985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83bce64-87e3-465c-b8ca-fc63f1d0f93a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:18.509324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83bce64-87e3-465c-b8ca-fc63f1d0f93a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.510817 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83bce64-87e3-465c-b8ca-fc63f1d0f93a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.524310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 255/519] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:18.525217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bce64-87e3-465c-b8ca-fc63f1d0f93a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.531079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578ad6c-6439-4d87-91e6-d22b6854a38a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.572498 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4a259-1c23-4971-895b-b7ea0e7bd535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.573097 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4a259-1c23-4971-895b-b7ea0e7bd535 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.573389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4a259-1c23-4971-895b-b7ea0e7bd535 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.573671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4a259-1c23-4971-895b-b7ea0e7bd535 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.574403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4a259-1c23-4971-895b-b7ea0e7bd535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:18.574711 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4a259-1c23-4971-895b-b7ea0e7bd535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.576123 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4a259-1c23-4971-895b-b7ea0e7bd535 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.587794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:18.589569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 265/520] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:18.591427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4a259-1c23-4971-895b-b7ea0e7bd535 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.638906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bce64-87e3-465c-b8ca-fc63f1d0f93a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.672780 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f295fc-686d-4c5e-a384-165a1583f875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.673426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f295fc-686d-4c5e-a384-165a1583f875 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.673715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f295fc-686d-4c5e-a384-165a1583f875 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.674023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f295fc-686d-4c5e-a384-165a1583f875 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.694215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4a259-1c23-4971-895b-b7ea0e7bd535 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.730107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64feec-1bab-430c-b66a-a9925526fb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.730775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64feec-1bab-430c-b66a-a9925526fb57 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.731062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64feec-1bab-430c-b66a-a9925526fb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.731345 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64feec-1bab-430c-b66a-a9925526fb57 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.732076 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d64feec-1bab-430c-b66a-a9925526fb57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:18.732420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d64feec-1bab-430c-b66a-a9925526fb57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.733850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d64feec-1bab-430c-b66a-a9925526fb57 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.740411 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:18.743761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f295fc-686d-4c5e-a384-165a1583f875 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.745338 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f295fc-686d-4c5e-a384-165a1583f875 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.750830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 266/521] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:18.750830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64feec-1bab-430c-b66a-a9925526fb57 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.805307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:18.851561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64feec-1bab-430c-b66a-a9925526fb57 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.853234 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 256/522] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:18.854053 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f295fc-686d-4c5e-a384-165a1583f875 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.889128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd76624-7798-4458-b8c7-6a48b17ab6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.889816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd76624-7798-4458-b8c7-6a48b17ab6bc None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.890105 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd76624-7798-4458-b8c7-6a48b17ab6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.890386 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd76624-7798-4458-b8c7-6a48b17ab6bc None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.891178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd76624-7798-4458-b8c7-6a48b17ab6bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:18.891961 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd76624-7798-4458-b8c7-6a48b17ab6bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.892964 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd76624-7798-4458-b8c7-6a48b17ab6bc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.911910 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 267/523] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:18.912827 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd76624-7798-4458-b8c7-6a48b17ab6bc None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.966149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f295fc-686d-4c5e-a384-165a1583f875 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.975749 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:19.000707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef251a-3baa-4b2a-82b0-4ccbb14b9373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.001387 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef251a-3baa-4b2a-82b0-4ccbb14b9373 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.001674 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef251a-3baa-4b2a-82b0-4ccbb14b9373 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.001952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef251a-3baa-4b2a-82b0-4ccbb14b9373 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.002657 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef251a-3baa-4b2a-82b0-4ccbb14b9373 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.004665 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef251a-3baa-4b2a-82b0-4ccbb14b9373 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:19.018134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd76624-7798-4458-b8c7-6a48b17ab6bc None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:19.052510 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fc165-5a6c-49e2-b875-eaf7afc574f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.053312 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fc165-5a6c-49e2-b875-eaf7afc574f7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.053602 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fc165-5a6c-49e2-b875-eaf7afc574f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.053883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fc165-5a6c-49e2-b875-eaf7afc574f7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.061216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fc165-5a6c-49e2-b875-eaf7afc574f7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.062677 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fc165-5a6c-49e2-b875-eaf7afc574f7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3b6fc165-5a6c-49e2-b875-eaf7afc574f7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:19.078359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:19.079469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 268/524] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:19.080513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fc165-5a6c-49e2-b875-eaf7afc574f7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:19.148104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 257/525] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:51:18 2022] POST /identity/v3/users => generated 247 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:19.148906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef251a-3baa-4b2a-82b0-4ccbb14b9373 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:19.165976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:19.196076 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fc165-5a6c-49e2-b875-eaf7afc574f7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:19.217559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:19.237252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ee2a2-982e-4663-b8cd-18a8bb9200e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.239393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ee2a2-982e-4663-b8cd-18a8bb9200e9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.239393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ee2a2-982e-4663-b8cd-18a8bb9200e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.239580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ee2a2-982e-4663-b8cd-18a8bb9200e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.247770 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ee2a2-982e-4663-b8cd-18a8bb9200e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.249728 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ee2a2-982e-4663-b8cd-18a8bb9200e9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-d28ee2a2-982e-4663-b8cd-18a8bb9200e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:19.266333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:19.267480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:19.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:19.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:19.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:19.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:19.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:19.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:19.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:19.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:19.268381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 269/526] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:19.269438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ee2a2-982e-4663-b8cd-18a8bb9200e9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:19.278063 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef251a-3baa-4b2a-82b0-4ccbb14b9373 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:19.320620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a01bf6-bf4e-43e5-9ac1-3613a363628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.321479 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a01bf6-bf4e-43e5-9ac1-3613a363628c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.321770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a01bf6-bf4e-43e5-9ac1-3613a363628c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.322057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a01bf6-bf4e-43e5-9ac1-3613a363628c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.334774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a01bf6-bf4e-43e5-9ac1-3613a363628c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.336095 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a01bf6-bf4e-43e5-9ac1-3613a363628c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-36a01bf6-bf4e-43e5-9ac1-3613a363628c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:19.356393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:19.357421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:19.358510 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:19.358510 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:19.358510 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:19.358510 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:19.358510 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 258/527] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:19.358856 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a01bf6-bf4e-43e5-9ac1-3613a363628c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:19.395424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ee2a2-982e-4663-b8cd-18a8bb9200e9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:19.429349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:19.434431 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d632b9-edd5-474c-b759-79c838afdbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.435093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d632b9-edd5-474c-b759-79c838afdbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.435385 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d632b9-edd5-474c-b759-79c838afdbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.435669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d632b9-edd5-474c-b759-79c838afdbc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.436555 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d632b9-edd5-474c-b759-79c838afdbc4 None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=5368284e42d84e77a6fdb88c2e8dcd3d) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:19.436876 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d632b9-edd5-474c-b759-79c838afdbc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.438356 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d632b9-edd5-474c-b759-79c838afdbc4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:19.445859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 270/528] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:19 2022] GET /identity/v3/role_assignments?user.id=5368284e42d84e77a6fdb88c2e8dcd3d&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:19.446703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d632b9-edd5-474c-b759-79c838afdbc4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:19.488050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a01bf6-bf4e-43e5-9ac1-3613a363628c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:19.529280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:19.545906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dd8ae-0359-4c33-aa0d-f9b7a783959e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.550340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dd8ae-0359-4c33-aa0d-f9b7a783959e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.550340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dd8ae-0359-4c33-aa0d-f9b7a783959e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.550340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dd8ae-0359-4c33-aa0d-f9b7a783959e None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.550340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dd8ae-0359-4c33-aa0d-f9b7a783959e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:19.550340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dd8ae-0359-4c33-aa0d-f9b7a783959e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.551979 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dd8ae-0359-4c33-aa0d-f9b7a783959e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:19.580498 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 259/529] 10.222.0.88 () {56 vars in 1139 bytes} [Thu Jun 16 14:51:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:19.583812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dd8ae-0359-4c33-aa0d-f9b7a783959e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:19.603894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d632b9-edd5-474c-b759-79c838afdbc4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:19.648013 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b97c7c-a632-4826-9942-371cf7f1aca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.648846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b97c7c-a632-4826-9942-371cf7f1aca6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.649342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b97c7c-a632-4826-9942-371cf7f1aca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.649618 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b97c7c-a632-4826-9942-371cf7f1aca6 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.650871 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b97c7c-a632-4826-9942-371cf7f1aca6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:19.651183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b97c7c-a632-4826-9942-371cf7f1aca6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.653014 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b97c7c-a632-4826-9942-371cf7f1aca6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:19.673236 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 271/530] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:19.674100 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b97c7c-a632-4826-9942-371cf7f1aca6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:19.734288 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dd8ae-0359-4c33-aa0d-f9b7a783959e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:19.780992 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:19.798185 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4803d-d977-40b7-b8c7-8f120aed896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.799403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4803d-d977-40b7-b8c7-8f120aed896a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.799946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4803d-d977-40b7-b8c7-8f120aed896a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.800947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4803d-d977-40b7-b8c7-8f120aed896a None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.802166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4803d-d977-40b7-b8c7-8f120aed896a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:19.803322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4803d-d977-40b7-b8c7-8f120aed896a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.805531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4803d-d977-40b7-b8c7-8f120aed896a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:19.822904 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b97c7c-a632-4826-9942-371cf7f1aca6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:19.823155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 260/531] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:19.824139 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4803d-d977-40b7-b8c7-8f120aed896a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:19.825114 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:19.862519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b255aa-ccd7-43f1-8d3d-a8eea361fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.863571 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b255aa-ccd7-43f1-8d3d-a8eea361fade None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.863965 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b255aa-ccd7-43f1-8d3d-a8eea361fade None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.864387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b255aa-ccd7-43f1-8d3d-a8eea361fade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.950400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4803d-d977-40b7-b8c7-8f120aed896a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:19.966544 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b255aa-ccd7-43f1-8d3d-a8eea361fade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.966748 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b255aa-ccd7-43f1-8d3d-a8eea361fade None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:19.991960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d29b6-ec84-420d-bc21-78dda7f832a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:19.992743 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d29b6-ec84-420d-bc21-78dda7f832a8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:19.993134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d29b6-ec84-420d-bc21-78dda7f832a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:19.993518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d29b6-ec84-420d-bc21-78dda7f832a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:19.994497 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d29b6-ec84-420d-bc21-78dda7f832a8 None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=e7ffcec55ac14f5bb8c3af6be2d4c7c6) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:19.994951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d29b6-ec84-420d-bc21-78dda7f832a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:19.996548 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d29b6-ec84-420d-bc21-78dda7f832a8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:20.008752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 261/532] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:19 2022] GET /identity/v3/role_assignments?user.id=e7ffcec55ac14f5bb8c3af6be2d4c7c6&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:20.009948 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d29b6-ec84-420d-bc21-78dda7f832a8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:20.090777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 272/533] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:20.096180 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b255aa-ccd7-43f1-8d3d-a8eea361fade None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:20.100599 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:20.165549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:20.212864 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b255aa-ccd7-43f1-8d3d-a8eea361fade None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:20.256740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ddbc3-535d-4566-a09a-95d1e29cd626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:20.257459 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ddbc3-535d-4566-a09a-95d1e29cd626 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:20.258003 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ddbc3-535d-4566-a09a-95d1e29cd626 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:20.258282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ddbc3-535d-4566-a09a-95d1e29cd626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:20.280130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ddbc3-535d-4566-a09a-95d1e29cd626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:20.281640 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ddbc3-535d-4566-a09a-95d1e29cd626 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:20.299672 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d29b6-ec84-420d-bc21-78dda7f832a8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-955ddbc3-535d-4566-a09a-95d1e29cd626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:20.319320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:20.320280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:20.321567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:20.321567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:20.321567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:20.321567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:20.321567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 273/534] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:20.329250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ddbc3-535d-4566-a09a-95d1e29cd626 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:20.352887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a5a64-97e6-41aa-9106-4b5b9f4c5e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:20.353694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5a64-97e6-41aa-9106-4b5b9f4c5e85 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:20.353979 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5a64-97e6-41aa-9106-4b5b9f4c5e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:20.354258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5a64-97e6-41aa-9106-4b5b9f4c5e85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:20.363624 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a5a64-97e6-41aa-9106-4b5b9f4c5e85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:20.365318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a5a64-97e6-41aa-9106-4b5b9f4c5e85 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1d9a5a64-97e6-41aa-9106-4b5b9f4c5e85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:20.382962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:20.383905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:20.386087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:20.386087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:20.386087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:20.386087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:20.386087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:20.386087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:20.386087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:20.386087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:20.386087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 262/535] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:20.393394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a5a64-97e6-41aa-9106-4b5b9f4c5e85 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:20.431405 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:20.525418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ddbc3-535d-4566-a09a-95d1e29cd626 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:20.564381 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a5a64-97e6-41aa-9106-4b5b9f4c5e85 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:20.582984 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:20.599793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b57971-6939-41e5-8068-cd6ed22fc913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:20.600564 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b57971-6939-41e5-8068-cd6ed22fc913 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:20.600850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b57971-6939-41e5-8068-cd6ed22fc913 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:20.601127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b57971-6939-41e5-8068-cd6ed22fc913 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:20.601857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b57971-6939-41e5-8068-cd6ed22fc913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:20.602163 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b57971-6939-41e5-8068-cd6ed22fc913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:20.603803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b57971-6939-41e5-8068-cd6ed22fc913 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:20.619613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 274/536] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:20.624876 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b57971-6939-41e5-8068-cd6ed22fc913 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:20.645104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a0358-9c29-465b-9c5a-d7ec08cbc6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:20.645784 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a0358-9c29-465b-9c5a-d7ec08cbc6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:20.646077 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a0358-9c29-465b-9c5a-d7ec08cbc6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:20.646365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a0358-9c29-465b-9c5a-d7ec08cbc6b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:20.647290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a0358-9c29-465b-9c5a-d7ec08cbc6b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:20.647613 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a0358-9c29-465b-9c5a-d7ec08cbc6b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:20.649126 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a0358-9c29-465b-9c5a-d7ec08cbc6b9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:20.686842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 263/537] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:20.694926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a0358-9c29-465b-9c5a-d7ec08cbc6b9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:20.736541 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b57971-6939-41e5-8068-cd6ed22fc913 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:20.764624 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:20.782028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec54231-7666-4609-a03b-7873f424e374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:20.782028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec54231-7666-4609-a03b-7873f424e374 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:20.782028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec54231-7666-4609-a03b-7873f424e374 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:20.782028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec54231-7666-4609-a03b-7873f424e374 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:20.787914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec54231-7666-4609-a03b-7873f424e374 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:20.789555 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec54231-7666-4609-a03b-7873f424e374 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-2ec54231-7666-4609-a03b-7873f424e374 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:20.807324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:20.808220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:20.809126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:20.809126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:20.809126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:20.809126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:20.809126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:20.809126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:20.809126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:20.809126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:20.809126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 275/538] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:20.816341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec54231-7666-4609-a03b-7873f424e374 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:20.925551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec54231-7666-4609-a03b-7873f424e374 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:20.958111 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a0358-9c29-465b-9c5a-d7ec08cbc6b9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:20.961644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560d68-2e1b-45a7-a2c4-2f905bc61641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:20.962378 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560d68-2e1b-45a7-a2c4-2f905bc61641 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:20.962734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560d68-2e1b-45a7-a2c4-2f905bc61641 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:20.963105 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560d68-2e1b-45a7-a2c4-2f905bc61641 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:20.963965 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42560d68-2e1b-45a7-a2c4-2f905bc61641 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:20.964375 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42560d68-2e1b-45a7-a2c4-2f905bc61641 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:20.966218 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42560d68-2e1b-45a7-a2c4-2f905bc61641 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:20.983479 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 276/539] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:20.996789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560d68-2e1b-45a7-a2c4-2f905bc61641 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:21.029924 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d98fa9-000a-4efb-89a1-f75278534de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:21.030716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d98fa9-000a-4efb-89a1-f75278534de0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:21.031006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d98fa9-000a-4efb-89a1-f75278534de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:21.031288 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d98fa9-000a-4efb-89a1-f75278534de0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:21.032181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d98fa9-000a-4efb-89a1-f75278534de0 None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=b39d53d1def347b6b36067e3e93aeb98) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:21.032515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d98fa9-000a-4efb-89a1-f75278534de0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:21.033966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d98fa9-000a-4efb-89a1-f75278534de0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:21.048897 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 264/540] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:20 2022] GET /identity/v3/role_assignments?user.id=b39d53d1def347b6b36067e3e93aeb98&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 259 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:21.068794 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:21.154755 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560d68-2e1b-45a7-a2c4-2f905bc61641 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:21.197471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fefeb-af3d-441a-8f8d-07cca9d4af4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:21.198154 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fefeb-af3d-441a-8f8d-07cca9d4af4a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:21.198436 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fefeb-af3d-441a-8f8d-07cca9d4af4a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:21.198714 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fefeb-af3d-441a-8f8d-07cca9d4af4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:21.199651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fefeb-af3d-441a-8f8d-07cca9d4af4a None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=58a2b345fe914d2cb97756e6fdf30010) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:21.199974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fefeb-af3d-441a-8f8d-07cca9d4af4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:21.201777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fefeb-af3d-441a-8f8d-07cca9d4af4a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:21.209567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 277/541] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:20 2022] GET /identity/v3/role_assignments?user.id=58a2b345fe914d2cb97756e6fdf30010&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:21.562977 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279144ca-3bb3-49e4-90dc-917c8dd0485b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:21.563526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279144ca-3bb3-49e4-90dc-917c8dd0485b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:21.563526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279144ca-3bb3-49e4-90dc-917c8dd0485b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:21.566203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 265/542] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:21.573096 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94821edc-0884-4242-acc4-e534e2ab3ddd None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:21.573574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94821edc-0884-4242-acc4-e534e2ab3ddd None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:21.573648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94821edc-0884-4242-acc4-e534e2ab3ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:21.592894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-94821edc-0884-4242-acc4-e534e2ab3ddd None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:21.594102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-94821edc-0884-4242-acc4-e534e2ab3ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:21.668080 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 278/543] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:21.693474 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279144ca-3bb3-49e4-90dc-917c8dd0485b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:21.727271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:21.823900 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279144ca-3bb3-49e4-90dc-917c8dd0485b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:21.874513 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0ca8c-2c05-4345-9fba-7d35f1dd8bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:21.875692 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0ca8c-2c05-4345-9fba-7d35f1dd8bbe None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:21.876176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0ca8c-2c05-4345-9fba-7d35f1dd8bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:21.876673 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0ca8c-2c05-4345-9fba-7d35f1dd8bbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:21.895775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0ca8c-2c05-4345-9fba-7d35f1dd8bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-cea0ca8c-2c05-4345-9fba-7d35f1dd8bbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:21.904149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:21.905065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:21.905874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:21.908252 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 266/544] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:21.915750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94821edc-0884-4242-acc4-e534e2ab3ddd None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:21.963086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:21.968370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ea6ec-d666-43e3-947b-cf43c84fb92c None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:21.968920 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ea6ec-d666-43e3-947b-cf43c84fb92c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:21.975806 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ea6ec-d666-43e3-947b-cf43c84fb92c None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:21.975806 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 267/545] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:51:21.985065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a42a1-90b6-4529-9475-5def7762d3ef None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:21.985598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a42a1-90b6-4529-9475-5def7762d3ef None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:21.986056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a42a1-90b6-4529-9475-5def7762d3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:22.022131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-819a42a1-90b6-4529-9475-5def7762d3ef None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:22.023317 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-819a42a1-90b6-4529-9475-5def7762d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:22.074799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94821edc-0884-4242-acc4-e534e2ab3ddd None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:22.118097 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18da3a-eb61-4d9d-b61c-2cb97ace88a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:22.118767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18da3a-eb61-4d9d-b61c-2cb97ace88a0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:22.119055 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18da3a-eb61-4d9d-b61c-2cb97ace88a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:22.119336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18da3a-eb61-4d9d-b61c-2cb97ace88a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:22.120086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18da3a-eb61-4d9d-b61c-2cb97ace88a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:22.120420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18da3a-eb61-4d9d-b61c-2cb97ace88a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:22.121807 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18da3a-eb61-4d9d-b61c-2cb97ace88a0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:22.143217 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 279/546] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:22.144786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 268/547] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:51:22.148864 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18da3a-eb61-4d9d-b61c-2cb97ace88a0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:22.186179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a42a1-90b6-4529-9475-5def7762d3ef None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:22.210555 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:22.317919 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:22.330220 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a42a1-90b6-4529-9475-5def7762d3ef None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:22.342444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18da3a-eb61-4d9d-b61c-2cb97ace88a0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:22.373678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf032b-98d2-4401-a4af-394b7d0830d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:22.374474 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf032b-98d2-4401-a4af-394b7d0830d7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:22.374755 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf032b-98d2-4401-a4af-394b7d0830d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:22.375028 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf032b-98d2-4401-a4af-394b7d0830d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:22.395441 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf032b-98d2-4401-a4af-394b7d0830d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-70cf032b-98d2-4401-a4af-394b7d0830d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:22.407375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:22.408916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:22.409962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 269/548] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:22.415358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf032b-98d2-4401-a4af-394b7d0830d7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:22.418689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b1862-491e-49c2-987e-f9f536771279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:22.419526 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b1862-491e-49c2-987e-f9f536771279 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:22.419810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b1862-491e-49c2-987e-f9f536771279 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:22.420094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b1862-491e-49c2-987e-f9f536771279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:22.432620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b1862-491e-49c2-987e-f9f536771279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:22.434289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b1862-491e-49c2-987e-f9f536771279 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-d05b1862-491e-49c2-987e-f9f536771279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:22.454472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:22.456594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:22.457933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:22.457933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:22.457933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:22.457933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:22.457933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 280/549] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:22.466468 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b1862-491e-49c2-987e-f9f536771279 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:22.521769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:22.573053 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf032b-98d2-4401-a4af-394b7d0830d7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:22.640942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b1862-491e-49c2-987e-f9f536771279 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:22.646056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74137c-5e54-44f8-b575-42b7e5fabf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:22.646805 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74137c-5e54-44f8-b575-42b7e5fabf7f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:22.647093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74137c-5e54-44f8-b575-42b7e5fabf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:22.647425 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74137c-5e54-44f8-b575-42b7e5fabf7f None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:22.648194 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74137c-5e54-44f8-b575-42b7e5fabf7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:22.648548 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74137c-5e54-44f8-b575-42b7e5fabf7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:22.649951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74137c-5e54-44f8-b575-42b7e5fabf7f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:22.671062 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 270/550] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:22.673553 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74137c-5e54-44f8-b575-42b7e5fabf7f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:22.713647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56c477-fa15-4219-b601-21f380d2d745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:22.714339 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c477-fa15-4219-b601-21f380d2d745 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:22.714629 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c477-fa15-4219-b601-21f380d2d745 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:22.714916 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c477-fa15-4219-b601-21f380d2d745 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:22.715660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56c477-fa15-4219-b601-21f380d2d745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:22.715976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56c477-fa15-4219-b601-21f380d2d745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:22.722671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56c477-fa15-4219-b601-21f380d2d745 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:22.751062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 281/551] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:22.757784 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56c477-fa15-4219-b601-21f380d2d745 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:22.764625 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:22.844997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74137c-5e54-44f8-b575-42b7e5fabf7f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:22.882975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1347cf-0ad5-4bf7-b9a5-ed7cf4371b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:22.887818 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1347cf-0ad5-4bf7-b9a5-ed7cf4371b02 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:22.888120 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1347cf-0ad5-4bf7-b9a5-ed7cf4371b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:22.888440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1347cf-0ad5-4bf7-b9a5-ed7cf4371b02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:22.907362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1347cf-0ad5-4bf7-b9a5-ed7cf4371b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:22.908811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1347cf-0ad5-4bf7-b9a5-ed7cf4371b02 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:22.911560 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:22.911854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56c477-fa15-4219-b601-21f380d2d745 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3c1347cf-0ad5-4bf7-b9a5-ed7cf4371b02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:22.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:22.935698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:22.936664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:22.936664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:22.936664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:22.936664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:22.936664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 271/552] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:22.936664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6eee2-85c1-4c02-be2b-19c43113a5cf None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:22.936664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6eee2-85c1-4c02-be2b-19c43113a5cf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:22.936664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6eee2-85c1-4c02-be2b-19c43113a5cf None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:22.937429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 272/553] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:22.939142 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6eee2-85c1-4c02-be2b-19c43113a5cf None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:22.971009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857c709-1dca-4ef0-8d56-94799a360a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:22.972679 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c709-1dca-4ef0-8d56-94799a360a04 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:22.973230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c709-1dca-4ef0-8d56-94799a360a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:22.973719 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c709-1dca-4ef0-8d56-94799a360a04 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:22.984269 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857c709-1dca-4ef0-8d56-94799a360a04 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:22.986663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857c709-1dca-4ef0-8d56-94799a360a04 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-6857c709-1dca-4ef0-8d56-94799a360a04 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:23.006136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:23.007370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:23.008874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:23.012357 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 282/554] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:23.013865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e970b50-6f9d-4aa2-a935-1e9056736493 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.014337 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e970b50-6f9d-4aa2-a935-1e9056736493 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.014624 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e970b50-6f9d-4aa2-a935-1e9056736493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.039970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2e970b50-6f9d-4aa2-a935-1e9056736493 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:23.041550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2e970b50-6f9d-4aa2-a935-1e9056736493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:23.090386 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6eee2-85c1-4c02-be2b-19c43113a5cf None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:23.111042 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:23.129205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da35ca0-c9e1-4209-a361-6f55925495ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:23.133906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da35ca0-c9e1-4209-a361-6f55925495ee None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.133906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da35ca0-c9e1-4209-a361-6f55925495ee None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.133906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da35ca0-c9e1-4209-a361-6f55925495ee None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.133906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da35ca0-c9e1-4209-a361-6f55925495ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:23.133906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da35ca0-c9e1-4209-a361-6f55925495ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:23.134243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 283/555] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:23.134330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da35ca0-c9e1-4209-a361-6f55925495ee None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:23.134402 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e970b50-6f9d-4aa2-a935-1e9056736493 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:23.156305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 273/556] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:23.163167 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da35ca0-c9e1-4209-a361-6f55925495ee None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:23.247609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:23.283026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e970b50-6f9d-4aa2-a935-1e9056736493 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:23.310881 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da35ca0-c9e1-4209-a361-6f55925495ee None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:23.331369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c78bd-1f21-4617-8ac3-aa54ac5d0d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:23.332866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c78bd-1f21-4617-8ac3-aa54ac5d0d29 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.333586 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c78bd-1f21-4617-8ac3-aa54ac5d0d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.334023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c78bd-1f21-4617-8ac3-aa54ac5d0d29 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.335253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c78bd-1f21-4617-8ac3-aa54ac5d0d29 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:23.335737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c78bd-1f21-4617-8ac3-aa54ac5d0d29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:23.337739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c78bd-1f21-4617-8ac3-aa54ac5d0d29 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:23.358530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef431c0-471b-4176-aa63-d388e16446fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:23.359517 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef431c0-471b-4176-aa63-d388e16446fe None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.360192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef431c0-471b-4176-aa63-d388e16446fe None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.360619 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef431c0-471b-4176-aa63-d388e16446fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.368061 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 284/557] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 14:51:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:23.368061 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c78bd-1f21-4617-8ac3-aa54ac5d0d29 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:23.375345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef431c0-471b-4176-aa63-d388e16446fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:23.377076 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef431c0-471b-4176-aa63-d388e16446fe None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1ef431c0-471b-4176-aa63-d388e16446fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:23.393917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:23.395450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:23.396469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:23.403490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 274/558] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 14:51:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:23.404730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef431c0-471b-4176-aa63-d388e16446fe None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:23.429647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:23.491372 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c78bd-1f21-4617-8ac3-aa54ac5d0d29 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:23.514034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef431c0-471b-4176-aa63-d388e16446fe None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:23.530883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:23.536900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a413e-b470-4f6c-842b-535d6a7c0d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:23.537885 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a413e-b470-4f6c-842b-535d6a7c0d1e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.538348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a413e-b470-4f6c-842b-535d6a7c0d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.538954 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a413e-b470-4f6c-842b-535d6a7c0d1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.550811 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a413e-b470-4f6c-842b-535d6a7c0d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:23.553475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76149e95-0e72-40fc-976e-0b812ae2399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:23.554233 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76149e95-0e72-40fc-976e-0b812ae2399c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.554594 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76149e95-0e72-40fc-976e-0b812ae2399c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.554952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76149e95-0e72-40fc-976e-0b812ae2399c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ee3a413e-b470-4f6c-842b-535d6a7c0d1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:23.561441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:23.562596 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:23.564070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:23.572266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 285/559] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:23.572266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a413e-b470-4f6c-842b-535d6a7c0d1e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:23.659195 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76149e95-0e72-40fc-976e-0b812ae2399c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:23.661491 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76149e95-0e72-40fc-976e-0b812ae2399c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:23.700854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a413e-b470-4f6c-842b-535d6a7c0d1e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:23.712885 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:23.740986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e72c91-9633-4d81-8ab4-cdcea5687b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:23.741658 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e72c91-9633-4d81-8ab4-cdcea5687b98 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.741946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e72c91-9633-4d81-8ab4-cdcea5687b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.742228 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e72c91-9633-4d81-8ab4-cdcea5687b98 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.743069 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e72c91-9633-4d81-8ab4-cdcea5687b98 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:23.743384 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e72c91-9633-4d81-8ab4-cdcea5687b98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:23.744872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e72c91-9633-4d81-8ab4-cdcea5687b98 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:23.764751 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 286/560] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 14:51:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:23.765590 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e72c91-9633-4d81-8ab4-cdcea5687b98 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:23.780774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 275/561] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:23.780774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67cb17-7cc4-428a-9b1b-ce08610f1836 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.780774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67cb17-7cc4-428a-9b1b-ce08610f1836 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.780774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67cb17-7cc4-428a-9b1b-ce08610f1836 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.780774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 276/562] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:23.780774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67cb17-7cc4-428a-9b1b-ce08610f1836 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:23.798112 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:23.883332 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e72c91-9633-4d81-8ab4-cdcea5687b98 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:23.890137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67cb17-7cc4-428a-9b1b-ce08610f1836 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:23.917766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb94ff-82ec-4750-b986-c45174955837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:23.918427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb94ff-82ec-4750-b986-c45174955837 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.918740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb94ff-82ec-4750-b986-c45174955837 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.919021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb94ff-82ec-4750-b986-c45174955837 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.919870 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb94ff-82ec-4750-b986-c45174955837 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:23.920178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb94ff-82ec-4750-b986-c45174955837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:23.921569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb94ff-82ec-4750-b986-c45174955837 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:23.923961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f148e52-b0e7-4ae9-b1dd-33c2acc81f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:23.929380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f148e52-b0e7-4ae9-b1dd-33c2acc81f17 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.929680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f148e52-b0e7-4ae9-b1dd-33c2acc81f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.929961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f148e52-b0e7-4ae9-b1dd-33c2acc81f17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.930331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 287/563] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:23.931108 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb94ff-82ec-4750-b986-c45174955837 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:23.941271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f148e52-b0e7-4ae9-b1dd-33c2acc81f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:23.943157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f148e52-b0e7-4ae9-b1dd-33c2acc81f17 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2f148e52-b0e7-4ae9-b1dd-33c2acc81f17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:23.962070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:23.963172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:23.968973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:23.968973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:23.968973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:23.968973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:23.968973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 277/564] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:23.968973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd0926-aa56-4d1b-b58e-b212a6e59dfe None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:23.968973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd0926-aa56-4d1b-b58e-b212a6e59dfe None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:23.968973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd0926-aa56-4d1b-b58e-b212a6e59dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:23.969918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:23.981684 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-53bd0926-aa56-4d1b-b58e-b212a6e59dfe None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:23.984044 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-53bd0926-aa56-4d1b-b58e-b212a6e59dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:24.037247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb94ff-82ec-4750-b986-c45174955837 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.045306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:24.054100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 278/565] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:24.055408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cf66b-76a9-476a-b600-b22c01c68fc4 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.055704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cf66b-76a9-476a-b600-b22c01c68fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.055992 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cf66b-76a9-476a-b600-b22c01c68fc4 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.057279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 279/566] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:24.058064 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cf66b-76a9-476a-b600-b22c01c68fc4 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.083584 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab5312-7be6-4a41-b9f8-bb7e6d7be6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.084579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab5312-7be6-4a41-b9f8-bb7e6d7be6cb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.084967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab5312-7be6-4a41-b9f8-bb7e6d7be6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.085367 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab5312-7be6-4a41-b9f8-bb7e6d7be6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.095793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab5312-7be6-4a41-b9f8-bb7e6d7be6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:24.098247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab5312-7be6-4a41-b9f8-bb7e6d7be6cb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c0ab5312-7be6-4a41-b9f8-bb7e6d7be6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:24.115569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:24.116701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:24.117797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:24.117797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:24.117797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:24.117797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:24.117797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 288/567] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:24.117797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab5312-7be6-4a41-b9f8-bb7e6d7be6cb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.165320 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cf66b-76a9-476a-b600-b22c01c68fc4 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.199716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779aba7a-d317-4673-8496-46accfe48089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.200555 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779aba7a-d317-4673-8496-46accfe48089 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.200845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779aba7a-d317-4673-8496-46accfe48089 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.201122 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779aba7a-d317-4673-8496-46accfe48089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.202847 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:24.212156 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779aba7a-d317-4673-8496-46accfe48089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:24.213581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779aba7a-d317-4673-8496-46accfe48089 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:24.221456 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab5312-7be6-4a41-b9f8-bb7e6d7be6cb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-779aba7a-d317-4673-8496-46accfe48089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:24.231541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:24.232695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:24.233609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:24.233609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:24.233609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:24.233609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:24.233609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 280/568] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:24.233943 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fabd9f-a7fe-4c19-910c-65403cb1064c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.234237 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fabd9f-a7fe-4c19-910c-65403cb1064c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.234512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fabd9f-a7fe-4c19-910c-65403cb1064c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.252217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e6fabd9f-a7fe-4c19-910c-65403cb1064c None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:24.253936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e6fabd9f-a7fe-4c19-910c-65403cb1064c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:24.258981 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4faaf4-95b9-4fde-bf1d-0aa8143f8847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.259651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4faaf4-95b9-4fde-bf1d-0aa8143f8847 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.259940 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4faaf4-95b9-4fde-bf1d-0aa8143f8847 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.260270 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4faaf4-95b9-4fde-bf1d-0aa8143f8847 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.260965 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4faaf4-95b9-4fde-bf1d-0aa8143f8847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:24.261275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4faaf4-95b9-4fde-bf1d-0aa8143f8847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:24.262657 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4faaf4-95b9-4fde-bf1d-0aa8143f8847 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:24.275892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 289/569] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:24.276732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4faaf4-95b9-4fde-bf1d-0aa8143f8847 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.300468 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:24.323590 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 281/570] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:24.324455 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fabd9f-a7fe-4c19-910c-65403cb1064c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.380839 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4faaf4-95b9-4fde-bf1d-0aa8143f8847 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.415382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669972ad-1dbf-4058-a160-b8046606603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.416177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669972ad-1dbf-4058-a160-b8046606603f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.416502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669972ad-1dbf-4058-a160-b8046606603f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.416786 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669972ad-1dbf-4058-a160-b8046606603f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.427957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669972ad-1dbf-4058-a160-b8046606603f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:24.432934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fabd9f-a7fe-4c19-910c-65403cb1064c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-669972ad-1dbf-4058-a160-b8046606603f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:24.435765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:24.436907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:24.437781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 290/571] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:24.438472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669972ad-1dbf-4058-a160-b8046606603f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.444525 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:24.466930 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907547fd-ee43-4c2d-9363-85011ff44fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.467519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907547fd-ee43-4c2d-9363-85011ff44fde None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.467807 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907547fd-ee43-4c2d-9363-85011ff44fde None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.468093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907547fd-ee43-4c2d-9363-85011ff44fde None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.468934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907547fd-ee43-4c2d-9363-85011ff44fde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:24.469246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907547fd-ee43-4c2d-9363-85011ff44fde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:24.470619 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907547fd-ee43-4c2d-9363-85011ff44fde None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:24.483196 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 282/572] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:24.484379 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907547fd-ee43-4c2d-9363-85011ff44fde None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.528863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:24.547554 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669972ad-1dbf-4058-a160-b8046606603f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.582859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537557c-09cb-48c1-be23-a2bd6cff0d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.583562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537557c-09cb-48c1-be23-a2bd6cff0d86 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.583878 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537557c-09cb-48c1-be23-a2bd6cff0d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.584162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537557c-09cb-48c1-be23-a2bd6cff0d86 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.584926 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537557c-09cb-48c1-be23-a2bd6cff0d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:24.585241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537557c-09cb-48c1-be23-a2bd6cff0d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:24.586693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537557c-09cb-48c1-be23-a2bd6cff0d86 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:24.592003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907547fd-ee43-4c2d-9363-85011ff44fde None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.599464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 291/573] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:24.600317 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537557c-09cb-48c1-be23-a2bd6cff0d86 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.635532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcce89-6fa8-4d4e-a4e7-6b2de6c11c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.636354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcce89-6fa8-4d4e-a4e7-6b2de6c11c2d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.636639 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcce89-6fa8-4d4e-a4e7-6b2de6c11c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.636915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcce89-6fa8-4d4e-a4e7-6b2de6c11c2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.644462 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcce89-6fa8-4d4e-a4e7-6b2de6c11c2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:24.646022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcce89-6fa8-4d4e-a4e7-6b2de6c11c2d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:24.660462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-f1fcce89-6fa8-4d4e-a4e7-6b2de6c11c2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:24.661857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:24.663243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:24.664358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:24.664358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:24.664358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:24.664358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:24.664358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:24.664358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:24.664358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:24.664358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:24.668257 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 283/574] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:24.668384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcce89-6fa8-4d4e-a4e7-6b2de6c11c2d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.699710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537557c-09cb-48c1-be23-a2bd6cff0d86 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.736051 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7640e97-af87-45ed-8f8e-74232bce43ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.736801 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7640e97-af87-45ed-8f8e-74232bce43ec None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.737169 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7640e97-af87-45ed-8f8e-74232bce43ec None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.737542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7640e97-af87-45ed-8f8e-74232bce43ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.747785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7640e97-af87-45ed-8f8e-74232bce43ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c7640e97-af87-45ed-8f8e-74232bce43ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:24.755669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:24.756820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 292/575] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:24.757986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7640e97-af87-45ed-8f8e-74232bce43ec None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.767448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:24.772347 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcce89-6fa8-4d4e-a4e7-6b2de6c11c2d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.809849 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbabe0e-28d9-44a0-bb3e-0121d4b6851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.810518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbabe0e-28d9-44a0-bb3e-0121d4b6851f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.810802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbabe0e-28d9-44a0-bb3e-0121d4b6851f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.811079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbabe0e-28d9-44a0-bb3e-0121d4b6851f None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.811828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbabe0e-28d9-44a0-bb3e-0121d4b6851f None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:24.812137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbabe0e-28d9-44a0-bb3e-0121d4b6851f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:24.813530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbabe0e-28d9-44a0-bb3e-0121d4b6851f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:24.821579 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 284/576] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:24.822365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbabe0e-28d9-44a0-bb3e-0121d4b6851f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.864710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7640e97-af87-45ed-8f8e-74232bce43ec None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.887180 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:24.900200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef3807-c38d-40cf-8699-b9016b1b9a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.901351 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef3807-c38d-40cf-8699-b9016b1b9a65 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.901643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef3807-c38d-40cf-8699-b9016b1b9a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.901960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef3807-c38d-40cf-8699-b9016b1b9a65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.908647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef3807-c38d-40cf-8699-b9016b1b9a65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:24.910561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef3807-c38d-40cf-8699-b9016b1b9a65 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:24.921496 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbabe0e-28d9-44a0-bb3e-0121d4b6851f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-34ef3807-c38d-40cf-8699-b9016b1b9a65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:24.925111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:24.931048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 293/577] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:24.932250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef3807-c38d-40cf-8699-b9016b1b9a65 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:24.955793 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c8ada-d29e-4c35-8e73-4149cd56e575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:24.957394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c8ada-d29e-4c35-8e73-4149cd56e575 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:24.957681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c8ada-d29e-4c35-8e73-4149cd56e575 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:24.957956 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c8ada-d29e-4c35-8e73-4149cd56e575 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:24.965243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c8ada-d29e-4c35-8e73-4149cd56e575 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:24.966666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c8ada-d29e-4c35-8e73-4149cd56e575 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-321c8ada-d29e-4c35-8e73-4149cd56e575 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:24.981476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:24.983322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 285/578] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:24.984911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c8ada-d29e-4c35-8e73-4149cd56e575 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.009657 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:25.035299 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef3807-c38d-40cf-8699-b9016b1b9a65 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.069952 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b47d7d-2253-4ce1-9aa6-6a3f0b586d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.070615 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b47d7d-2253-4ce1-9aa6-6a3f0b586d70 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.070902 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b47d7d-2253-4ce1-9aa6-6a3f0b586d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.071191 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b47d7d-2253-4ce1-9aa6-6a3f0b586d70 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.071973 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b47d7d-2253-4ce1-9aa6-6a3f0b586d70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:25.072314 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b47d7d-2253-4ce1-9aa6-6a3f0b586d70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.073755 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b47d7d-2253-4ce1-9aa6-6a3f0b586d70 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.086299 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 294/579] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:25.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b47d7d-2253-4ce1-9aa6-6a3f0b586d70 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.089387 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c8ada-d29e-4c35-8e73-4149cd56e575 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.127502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e176bda-a2ad-4de3-92e5-53419efbfc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.128763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e176bda-a2ad-4de3-92e5-53419efbfc8f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.129159 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e176bda-a2ad-4de3-92e5-53419efbfc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.129474 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e176bda-a2ad-4de3-92e5-53419efbfc8f None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.130253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e176bda-a2ad-4de3-92e5-53419efbfc8f None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:25.130559 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e176bda-a2ad-4de3-92e5-53419efbfc8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.131947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e176bda-a2ad-4de3-92e5-53419efbfc8f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.136894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:25.140612 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 286/580] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:25.141423 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e176bda-a2ad-4de3-92e5-53419efbfc8f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.194870 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b47d7d-2253-4ce1-9aa6-6a3f0b586d70 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.226867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:25.228576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cd576-fc30-432d-ae04-261073a66d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.229378 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cd576-fc30-432d-ae04-261073a66d45 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.229666 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cd576-fc30-432d-ae04-261073a66d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.229946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cd576-fc30-432d-ae04-261073a66d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.241117 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cd576-fc30-432d-ae04-261073a66d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.245038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cd576-fc30-432d-ae04-261073a66d45 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.245138 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e176bda-a2ad-4de3-92e5-53419efbfc8f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-247cd576-fc30-432d-ae04-261073a66d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:25.260399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:25.261483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:25.262601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:25.262601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:25.262601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:25.262601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:25.262601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 295/581] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:25.262601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cd576-fc30-432d-ae04-261073a66d45 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.285462 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad090b-08f0-4650-865f-b1e22007e837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.286133 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad090b-08f0-4650-865f-b1e22007e837 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.286424 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad090b-08f0-4650-865f-b1e22007e837 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.286707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad090b-08f0-4650-865f-b1e22007e837 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.287557 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad090b-08f0-4650-865f-b1e22007e837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:25.288130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad090b-08f0-4650-865f-b1e22007e837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.290215 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad090b-08f0-4650-865f-b1e22007e837 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.306558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 287/582] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:25.306558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad090b-08f0-4650-865f-b1e22007e837 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.367673 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:25.374810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cd576-fc30-432d-ae04-261073a66d45 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.404729 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad090b-08f0-4650-865f-b1e22007e837 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.408792 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe030e-11cc-48d8-944e-da7d6a1919a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.410253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe030e-11cc-48d8-944e-da7d6a1919a8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.410564 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe030e-11cc-48d8-944e-da7d6a1919a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.410848 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe030e-11cc-48d8-944e-da7d6a1919a8 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.411692 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe030e-11cc-48d8-944e-da7d6a1919a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:25.412012 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe030e-11cc-48d8-944e-da7d6a1919a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.413548 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe030e-11cc-48d8-944e-da7d6a1919a8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.430501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 296/583] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 14:51:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:25.431313 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe030e-11cc-48d8-944e-da7d6a1919a8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.442703 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e823c-9db3-41e7-88b9-3e3545ccb040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.443597 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e823c-9db3-41e7-88b9-3e3545ccb040 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.443889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e823c-9db3-41e7-88b9-3e3545ccb040 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.444170 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e823c-9db3-41e7-88b9-3e3545ccb040 None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/5368284e42d84e77a6fdb88c2e8dcd3d/roles/b9c2d4c037024096bd7e864cea8b55e8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.456824 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:25.460075 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e823c-9db3-41e7-88b9-3e3545ccb040 None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=5368284e42d84e77a6fdb88c2e8dcd3d, role_id=b9c2d4c037024096bd7e864cea8b55e8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.464633 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e823c-9db3-41e7-88b9-3e3545ccb040 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.530731 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe030e-11cc-48d8-944e-da7d6a1919a8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.564850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d679a8-ce73-4ee5-b0c6-5ba42a979583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.565651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d679a8-ce73-4ee5-b0c6-5ba42a979583 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.565940 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d679a8-ce73-4ee5-b0c6-5ba42a979583 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.566220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d679a8-ce73-4ee5-b0c6-5ba42a979583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.576218 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d679a8-ce73-4ee5-b0c6-5ba42a979583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.577725 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d679a8-ce73-4ee5-b0c6-5ba42a979583 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.588009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 288/584] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:51:25 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/5368284e42d84e77a6fdb88c2e8dcd3d/roles/b9c2d4c037024096bd7e864cea8b55e8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:25.595132 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e823c-9db3-41e7-88b9-3e3545ccb040 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-40d679a8-ce73-4ee5-b0c6-5ba42a979583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:25.596021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:25.597950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:25.598902 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:25.598902 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:25.598902 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:25.598902 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:25.602373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 297/585] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:25.603301 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d679a8-ce73-4ee5-b0c6-5ba42a979583 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.607207 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:25.718872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d679a8-ce73-4ee5-b0c6-5ba42a979583 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.725048 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e823c-9db3-41e7-88b9-3e3545ccb040 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.759304 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8400212-779f-4514-86d5-dbb24d7dceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.759970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8400212-779f-4514-86d5-dbb24d7dceb1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.760300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8400212-779f-4514-86d5-dbb24d7dceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.760595 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8400212-779f-4514-86d5-dbb24d7dceb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.761446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8400212-779f-4514-86d5-dbb24d7dceb1 None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=bf0dd8c9980e4640aafd34946c4193a3) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:25.761764 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8400212-779f-4514-86d5-dbb24d7dceb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.763254 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8400212-779f-4514-86d5-dbb24d7dceb1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.770192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 298/586] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:25 2022] GET /identity/v3/role_assignments?user.id=bf0dd8c9980e4640aafd34946c4193a3&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:25.771020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8400212-779f-4514-86d5-dbb24d7dceb1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.777850 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfa5c0-842b-4182-8365-595c7942cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.778737 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfa5c0-842b-4182-8365-595c7942cb29 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.779031 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfa5c0-842b-4182-8365-595c7942cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.779407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfa5c0-842b-4182-8365-595c7942cb29 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.780071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfa5c0-842b-4182-8365-595c7942cb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:25.780426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfa5c0-842b-4182-8365-595c7942cb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.781869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfa5c0-842b-4182-8365-595c7942cb29 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.796571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 289/587] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:25.797455 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfa5c0-842b-4182-8365-595c7942cb29 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:25.868761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:25.871398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:25.917819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8400212-779f-4514-86d5-dbb24d7dceb1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.938440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfa5c0-842b-4182-8365-595c7942cb29 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:25.963808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c154c-8bf1-492f-914d-56891b53c532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.965383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c154c-8bf1-492f-914d-56891b53c532 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.965950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c154c-8bf1-492f-914d-56891b53c532 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.966455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c154c-8bf1-492f-914d-56891b53c532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.978748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d210d2-60fe-4f73-a097-904294712886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:25.979424 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d210d2-60fe-4f73-a097-904294712886 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:25.981148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d210d2-60fe-4f73-a097-904294712886 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:25.981148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d210d2-60fe-4f73-a097-904294712886 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:25.981148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d210d2-60fe-4f73-a097-904294712886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:25.981785 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d210d2-60fe-4f73-a097-904294712886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.983185 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d210d2-60fe-4f73-a097-904294712886 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.989401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c154c-8bf1-492f-914d-56891b53c532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:25.992187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c154c-8bf1-492f-914d-56891b53c532 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:25.998271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 290/588] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:25.999483 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d210d2-60fe-4f73-a097-904294712886 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ea1c154c-8bf1-492f-914d-56891b53c532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:26.016081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:26.017206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:26.018147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:26.018147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:26.018147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:26.018147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:26.018147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 299/589] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:26.018857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c154c-8bf1-492f-914d-56891b53c532 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.130503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d210d2-60fe-4f73-a097-904294712886 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.147582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:26.149507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c154c-8bf1-492f-914d-56891b53c532 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.171642 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7b72e-27b5-4358-9521-af447e23fb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.172793 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7b72e-27b5-4358-9521-af447e23fb46 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.172908 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7b72e-27b5-4358-9521-af447e23fb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.173136 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7b72e-27b5-4358-9521-af447e23fb46 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.176747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:26.182530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7b72e-27b5-4358-9521-af447e23fb46 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.184285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7b72e-27b5-4358-9521-af447e23fb46 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.196425 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf24478-c27e-4118-b9cc-22787a66ebd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.197223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf24478-c27e-4118-b9cc-22787a66ebd3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.197510 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf24478-c27e-4118-b9cc-22787a66ebd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.197790 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf24478-c27e-4118-b9cc-22787a66ebd3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ecd7b72e-27b5-4358-9521-af447e23fb46 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:26.203348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:26.204427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:51:26.205357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:26.206562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf24478-c27e-4118-b9cc-22787a66ebd3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.208102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf24478-c27e-4118-b9cc-22787a66ebd3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.209199 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 291/590] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:26.210260 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7b72e-27b5-4358-9521-af447e23fb46 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-4cf24478-c27e-4118-b9cc-22787a66ebd3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:26.233246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:26.234273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:51:26.235209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:26.239254 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 300/591] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:26.240332 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf24478-c27e-4118-b9cc-22787a66ebd3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.354205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7b72e-27b5-4358-9521-af447e23fb46 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.364468 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf24478-c27e-4118-b9cc-22787a66ebd3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.398867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaf8b3-0e2b-4674-9179-b4d597a4af46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.399454 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaf8b3-0e2b-4674-9179-b4d597a4af46 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.399742 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaf8b3-0e2b-4674-9179-b4d597a4af46 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.400022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaf8b3-0e2b-4674-9179-b4d597a4af46 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.400785 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aaf8b3-0e2b-4674-9179-b4d597a4af46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:26.401086 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aaf8b3-0e2b-4674-9179-b4d597a4af46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.402462 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aaf8b3-0e2b-4674-9179-b4d597a4af46 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.404398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db6f82-78aa-4314-b56e-13e625ef931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.405284 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db6f82-78aa-4314-b56e-13e625ef931a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.405767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db6f82-78aa-4314-b56e-13e625ef931a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.406241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db6f82-78aa-4314-b56e-13e625ef931a None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.407304 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db6f82-78aa-4314-b56e-13e625ef931a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:26.407825 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db6f82-78aa-4314-b56e-13e625ef931a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.410157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db6f82-78aa-4314-b56e-13e625ef931a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.414947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 292/592] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:26.415801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaf8b3-0e2b-4674-9179-b4d597a4af46 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.430889 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 301/593] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:26.432166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db6f82-78aa-4314-b56e-13e625ef931a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.450516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:26.477891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:26.536662 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaf8b3-0e2b-4674-9179-b4d597a4af46 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.545590 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db6f82-78aa-4314-b56e-13e625ef931a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.576543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2534f-d4bc-43c8-ac56-3cafae54ee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.577495 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2534f-d4bc-43c8-ac56-3cafae54ee3b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.577948 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2534f-d4bc-43c8-ac56-3cafae54ee3b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.578417 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2534f-d4bc-43c8-ac56-3cafae54ee3b None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.579625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2534f-d4bc-43c8-ac56-3cafae54ee3b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:26.580078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2534f-d4bc-43c8-ac56-3cafae54ee3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.581953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2534f-d4bc-43c8-ac56-3cafae54ee3b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.587088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30139bb2-61bc-44e4-9f98-a3aaea0e4a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.587088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30139bb2-61bc-44e4-9f98-a3aaea0e4a71 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.587088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30139bb2-61bc-44e4-9f98-a3aaea0e4a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.587088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30139bb2-61bc-44e4-9f98-a3aaea0e4a71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.593878 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30139bb2-61bc-44e4-9f98-a3aaea0e4a71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.595441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30139bb2-61bc-44e4-9f98-a3aaea0e4a71 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.603129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 293/594] 10.222.0.88 () {56 vars in 1139 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:26.604293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2534f-d4bc-43c8-ac56-3cafae54ee3b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-30139bb2-61bc-44e4-9f98-a3aaea0e4a71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:26.611446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:26.613204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:26.614164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:26.614164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:26.614164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:26.614164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:26.614164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:26.614164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:26.614164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:26.614164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:26.614164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 302/595] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:26.615685 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30139bb2-61bc-44e4-9f98-a3aaea0e4a71 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.709607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:26.716945 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:26.720668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30139bb2-61bc-44e4-9f98-a3aaea0e4a71 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.724569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2534f-d4bc-43c8-ac56-3cafae54ee3b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.756128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f258000-2c84-4ec6-84cc-2960855e5d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.757180 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f258000-2c84-4ec6-84cc-2960855e5d1d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.764488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f258000-2c84-4ec6-84cc-2960855e5d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.764488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f258000-2c84-4ec6-84cc-2960855e5d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.765097 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291d47c-51ec-4935-96ab-1619d1c1908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.765097 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291d47c-51ec-4935-96ab-1619d1c1908a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.765097 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291d47c-51ec-4935-96ab-1619d1c1908a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.765097 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291d47c-51ec-4935-96ab-1619d1c1908a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.770193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f258000-2c84-4ec6-84cc-2960855e5d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.771755 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f258000-2c84-4ec6-84cc-2960855e5d1d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.775181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291d47c-51ec-4935-96ab-1619d1c1908a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.777203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291d47c-51ec-4935-96ab-1619d1c1908a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-4f258000-2c84-4ec6-84cc-2960855e5d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:26.790661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:26.791612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:26.792551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:26.792551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:26.792551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:26.792551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:26.792551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 303/596] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:26.793586 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f258000-2c84-4ec6-84cc-2960855e5d1d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7291d47c-51ec-4935-96ab-1619d1c1908a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:26.799134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:26.800554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:26.801667 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:26.801667 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:26.801667 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:26.801667 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:26.803889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 294/597] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:26.804925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291d47c-51ec-4935-96ab-1619d1c1908a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.917784 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f258000-2c84-4ec6-84cc-2960855e5d1d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.924977 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291d47c-51ec-4935-96ab-1619d1c1908a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:26.959451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604d13d-8735-4cc7-8de5-0265eb1bbacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.960526 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604d13d-8735-4cc7-8de5-0265eb1bbacd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.960855 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604d13d-8735-4cc7-8de5-0265eb1bbacd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.961250 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604d13d-8735-4cc7-8de5-0265eb1bbacd None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.962143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604d13d-8735-4cc7-8de5-0265eb1bbacd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:26.962652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604d13d-8735-4cc7-8de5-0265eb1bbacd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.964417 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604d13d-8735-4cc7-8de5-0265eb1bbacd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.966027 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6f2f3-110f-40c1-929a-676bdee555bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:26.966949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6f2f3-110f-40c1-929a-676bdee555bc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:26.967497 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6f2f3-110f-40c1-929a-676bdee555bc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:26.967919 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6f2f3-110f-40c1-929a-676bdee555bc None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:26.968875 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6f2f3-110f-40c1-929a-676bdee555bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:26.969328 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6f2f3-110f-40c1-929a-676bdee555bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:26.970830 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6f2f3-110f-40c1-929a-676bdee555bc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:26.978789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 304/598] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:26.979758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604d13d-8735-4cc7-8de5-0265eb1bbacd None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.984663 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 295/599] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:26.985639 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6f2f3-110f-40c1-929a-676bdee555bc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:26.989461 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:26.994033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:27.097605 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6f2f3-110f-40c1-929a-676bdee555bc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:27.128705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604d13d-8735-4cc7-8de5-0265eb1bbacd None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:27.156121 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176351bd-fc43-4302-a59d-bf6259c24fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:27.158183 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176351bd-fc43-4302-a59d-bf6259c24fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:27.159106 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176351bd-fc43-4302-a59d-bf6259c24fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:27.159737 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176351bd-fc43-4302-a59d-bf6259c24fa3 None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/e7ffcec55ac14f5bb8c3af6be2d4c7c6/roles/b9c2d4c037024096bd7e864cea8b55e8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:27.170765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7800ae-27e0-457e-8f22-cb6102c1c290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:27.171597 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7800ae-27e0-457e-8f22-cb6102c1c290 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:27.172026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7800ae-27e0-457e-8f22-cb6102c1c290 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:27.172476 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7800ae-27e0-457e-8f22-cb6102c1c290 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:27.173344 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7800ae-27e0-457e-8f22-cb6102c1c290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:27.173800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7800ae-27e0-457e-8f22-cb6102c1c290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:27.175744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7800ae-27e0-457e-8f22-cb6102c1c290 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:27.185766 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176351bd-fc43-4302-a59d-bf6259c24fa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=e7ffcec55ac14f5bb8c3af6be2d4c7c6, role_id=b9c2d4c037024096bd7e864cea8b55e8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:27.187428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176351bd-fc43-4302-a59d-bf6259c24fa3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:27.199701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 305/600] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:27.204355 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7800ae-27e0-457e-8f22-cb6102c1c290 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:27.265187 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 296/601] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:51:26 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/e7ffcec55ac14f5bb8c3af6be2d4c7c6/roles/b9c2d4c037024096bd7e864cea8b55e8 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:27.266326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176351bd-fc43-4302-a59d-bf6259c24fa3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:27.285813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:27.338426 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7800ae-27e0-457e-8f22-cb6102c1c290 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:27.363634 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:27.424062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9da4d-84dd-456c-b588-1b7dc98794e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:27.425897 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9da4d-84dd-456c-b588-1b7dc98794e0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:27.426196 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9da4d-84dd-456c-b588-1b7dc98794e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:27.426479 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9da4d-84dd-456c-b588-1b7dc98794e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:27.443691 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9da4d-84dd-456c-b588-1b7dc98794e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:27.445234 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9da4d-84dd-456c-b588-1b7dc98794e0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-6ba9da4d-84dd-456c-b588-1b7dc98794e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:27.465467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:27.466466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:27.467368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:27.467368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:27.467368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:27.467368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:27.467368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:27.467368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:27.467368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:27.467368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:27.467368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 306/602] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:27.470266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176351bd-fc43-4302-a59d-bf6259c24fa3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:27.473684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9da4d-84dd-456c-b588-1b7dc98794e0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:27.516775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229b8ad-184e-43bd-9734-fe54e318a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:27.517669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b8ad-184e-43bd-9734-fe54e318a63b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:27.518058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b8ad-184e-43bd-9734-fe54e318a63b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:27.519058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229b8ad-184e-43bd-9734-fe54e318a63b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:27.526609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229b8ad-184e-43bd-9734-fe54e318a63b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:27.528839 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229b8ad-184e-43bd-9734-fe54e318a63b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-f229b8ad-184e-43bd-9734-fe54e318a63b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:27.547718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:27.548707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:27.549854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:27.549854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:27.549854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:27.549854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:27.549854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:27.549854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:27.549854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:27.549854 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:27.550290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 297/603] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:27.555668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229b8ad-184e-43bd-9734-fe54e318a63b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:27.599156 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9da4d-84dd-456c-b588-1b7dc98794e0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:27.654770 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:27.682838 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229b8ad-184e-43bd-9734-fe54e318a63b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:27.686139 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aff0ec-145e-46e4-aa16-0213d62f4adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:27.686867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aff0ec-145e-46e4-aa16-0213d62f4adf None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:27.687373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aff0ec-145e-46e4-aa16-0213d62f4adf None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:27.687850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aff0ec-145e-46e4-aa16-0213d62f4adf None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:27.688925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aff0ec-145e-46e4-aa16-0213d62f4adf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:27.689418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aff0ec-145e-46e4-aa16-0213d62f4adf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:27.692052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aff0ec-145e-46e4-aa16-0213d62f4adf None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:27.697179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:27.722572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc7770-5981-47cb-aa29-2903020b15fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:27.723454 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc7770-5981-47cb-aa29-2903020b15fa None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:27.723748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc7770-5981-47cb-aa29-2903020b15fa None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:27.724036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc7770-5981-47cb-aa29-2903020b15fa None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:27.724928 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc7770-5981-47cb-aa29-2903020b15fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:27.725573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc7770-5981-47cb-aa29-2903020b15fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:27.727039 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc7770-5981-47cb-aa29-2903020b15fa None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:27.727753 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 307/604] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:27.733475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aff0ec-145e-46e4-aa16-0213d62f4adf None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:27.743155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 298/605] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:27.749042 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc7770-5981-47cb-aa29-2903020b15fa None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:27.850644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aff0ec-145e-46e4-aa16-0213d62f4adf None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:27.879762 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc7770-5981-47cb-aa29-2903020b15fa None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:27.919783 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed1849-97a1-4dad-971e-a0f63e9a1c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:27.920733 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed1849-97a1-4dad-971e-a0f63e9a1c3b None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:27.921020 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed1849-97a1-4dad-971e-a0f63e9a1c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:27.921302 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed1849-97a1-4dad-971e-a0f63e9a1c3b None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/b39d53d1def347b6b36067e3e93aeb98/roles/b9c2d4c037024096bd7e864cea8b55e8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:27.938013 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed1849-97a1-4dad-971e-a0f63e9a1c3b None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=b39d53d1def347b6b36067e3e93aeb98, role_id=b9c2d4c037024096bd7e864cea8b55e8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:27.939560 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed1849-97a1-4dad-971e-a0f63e9a1c3b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:27.953519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fc527-4fc2-4bf0-9cbe-660fa468d36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:27.958482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fc527-4fc2-4bf0-9cbe-660fa468d36d None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:27.958482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fc527-4fc2-4bf0-9cbe-660fa468d36d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:27.958482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fc527-4fc2-4bf0-9cbe-660fa468d36d None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/58a2b345fe914d2cb97756e6fdf30010/roles/b9c2d4c037024096bd7e864cea8b55e8` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:27.973962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fc527-4fc2-4bf0-9cbe-660fa468d36d None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=58a2b345fe914d2cb97756e6fdf30010, role_id=b9c2d4c037024096bd7e864cea8b55e8)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:27.975757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fc527-4fc2-4bf0-9cbe-660fa468d36d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:28.065119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 299/606] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:51:27 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/b39d53d1def347b6b36067e3e93aeb98/roles/b9c2d4c037024096bd7e864cea8b55e8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:28.092781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 308/607] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:51:27 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/58a2b345fe914d2cb97756e6fdf30010/roles/b9c2d4c037024096bd7e864cea8b55e8 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:51:28.117932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b14c4-b3b4-4c1c-abbe-2c2e21e68cdf None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:28.118186 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b14c4-b3b4-4c1c-abbe-2c2e21e68cdf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:28.118427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b14c4-b3b4-4c1c-abbe-2c2e21e68cdf None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:28.120192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 300/608] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:28.125769 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439acfc-deaa-4f30-9bfa-6734e451988c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:28.126357 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439acfc-deaa-4f30-9bfa-6734e451988c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:28.126842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439acfc-deaa-4f30-9bfa-6734e451988c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:28.148899 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afae87d-ff6f-4538-8148-070da719e09b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:28.149209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afae87d-ff6f-4538-8148-070da719e09b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:28.149490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afae87d-ff6f-4538-8148-070da719e09b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:28.151223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 301/609] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:28.152120 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3439acfc-deaa-4f30-9bfa-6734e451988c None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:28.153988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3439acfc-deaa-4f30-9bfa-6734e451988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:28.157097 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d3023-0460-4dd0-b9ca-b1f134b291e5 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:28.157404 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d3023-0460-4dd0-b9ca-b1f134b291e5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:28.157679 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d3023-0460-4dd0-b9ca-b1f134b291e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:28.197943 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ee5d3023-0460-4dd0-b9ca-b1f134b291e5 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:28.199286 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ee5d3023-0460-4dd0-b9ca-b1f134b291e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:28.229975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:28.236767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:28.243680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 309/610] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:28.281597 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439acfc-deaa-4f30-9bfa-6734e451988c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:28.312965 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 302/611] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:51:28.352579 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d3023-0460-4dd0-b9ca-b1f134b291e5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:28.424372 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439acfc-deaa-4f30-9bfa-6734e451988c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:28.461397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d72d4-7a90-46de-980f-6f817d500217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:28.462204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d72d4-7a90-46de-980f-6f817d500217 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:28.462492 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d72d4-7a90-46de-980f-6f817d500217 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:28.462774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d72d4-7a90-46de-980f-6f817d500217 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:28.472725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d3023-0460-4dd0-b9ca-b1f134b291e5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:28.474889 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d72d4-7a90-46de-980f-6f817d500217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-687d72d4-7a90-46de-980f-6f817d500217 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:28.482988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:28.483944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:28.484838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 310/612] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:28.491993 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d72d4-7a90-46de-980f-6f817d500217 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:28.527884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dad7c8-cb81-4904-af79-172ba01959c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:28.528831 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad7c8-cb81-4904-af79-172ba01959c1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:28.529203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad7c8-cb81-4904-af79-172ba01959c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:28.529566 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad7c8-cb81-4904-af79-172ba01959c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:28.548585 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:28.555725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dad7c8-cb81-4904-af79-172ba01959c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d8dad7c8-cb81-4904-af79-172ba01959c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:28.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:28.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:28.573465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 303/613] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:28.579779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dad7c8-cb81-4904-af79-172ba01959c1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:28.612632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:28.679247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d72d4-7a90-46de-980f-6f817d500217 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:28.745714 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ada36-5e1f-4329-9b69-58093ab6bb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:28.746396 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ada36-5e1f-4329-9b69-58093ab6bb67 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:28.746686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ada36-5e1f-4329-9b69-58093ab6bb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:28.746966 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ada36-5e1f-4329-9b69-58093ab6bb67 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:28.747735 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ada36-5e1f-4329-9b69-58093ab6bb67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:28.748046 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ada36-5e1f-4329-9b69-58093ab6bb67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:28.749460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ada36-5e1f-4329-9b69-58093ab6bb67 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:28.765249 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 311/614] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:28.770679 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ada36-5e1f-4329-9b69-58093ab6bb67 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:28.785559 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dad7c8-cb81-4904-af79-172ba01959c1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:28.860670 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d330b-1c16-4827-9634-657dfc7b3683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:28.861405 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d330b-1c16-4827-9634-657dfc7b3683 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:28.861818 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d330b-1c16-4827-9634-657dfc7b3683 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:28.862747 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d330b-1c16-4827-9634-657dfc7b3683 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:28.863565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d330b-1c16-4827-9634-657dfc7b3683 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:28.863971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d330b-1c16-4827-9634-657dfc7b3683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:28.869617 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d330b-1c16-4827-9634-657dfc7b3683 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:28.886281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 304/615] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:28.893929 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d330b-1c16-4827-9634-657dfc7b3683 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:28.939317 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:28.948691 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ada36-5e1f-4329-9b69-58093ab6bb67 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:28.988779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df1958-ac6b-4f08-af49-dee7d8de03dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:28.991224 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df1958-ac6b-4f08-af49-dee7d8de03dd None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:28.991743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df1958-ac6b-4f08-af49-dee7d8de03dd None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:28.993284 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df1958-ac6b-4f08-af49-dee7d8de03dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:29.017657 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df1958-ac6b-4f08-af49-dee7d8de03dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:29.020286 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df1958-ac6b-4f08-af49-dee7d8de03dd None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:29.034352 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-12df1958-ac6b-4f08-af49-dee7d8de03dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:29.042352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:29.043363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:29.044359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:29.044359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:29.044359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:29.044359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:29.047435 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 312/616] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:29.052776 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df1958-ac6b-4f08-af49-dee7d8de03dd None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:29.094695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d330b-1c16-4827-9634-657dfc7b3683 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:29.165271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ff712-b1e6-4e47-8dfa-fd8eb541a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:29.166447 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ff712-b1e6-4e47-8dfa-fd8eb541a851 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:29.166954 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ff712-b1e6-4e47-8dfa-fd8eb541a851 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:29.167434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ff712-b1e6-4e47-8dfa-fd8eb541a851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:29.179874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ff712-b1e6-4e47-8dfa-fd8eb541a851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:29.181994 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ff712-b1e6-4e47-8dfa-fd8eb541a851 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ff2ff712-b1e6-4e47-8dfa-fd8eb541a851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:29.201341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:29.202429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:29.203342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:29.203342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:29.203342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:29.203342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:29.206825 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 305/617] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:29.212303 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ff712-b1e6-4e47-8dfa-fd8eb541a851 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:29.277232 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df1958-ac6b-4f08-af49-dee7d8de03dd None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:29.315418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cad340-b2a5-4bc1-8982-d4e58ee3d248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:29.316514 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cad340-b2a5-4bc1-8982-d4e58ee3d248 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:29.317059 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cad340-b2a5-4bc1-8982-d4e58ee3d248 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:29.317905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cad340-b2a5-4bc1-8982-d4e58ee3d248 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:29.319006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cad340-b2a5-4bc1-8982-d4e58ee3d248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:29.319533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cad340-b2a5-4bc1-8982-d4e58ee3d248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:29.321167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cad340-b2a5-4bc1-8982-d4e58ee3d248 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:29.335362 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 313/618] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:29.343423 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cad340-b2a5-4bc1-8982-d4e58ee3d248 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:29.356240 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:29.449538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ff712-b1e6-4e47-8dfa-fd8eb541a851 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:29.494389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cad340-b2a5-4bc1-8982-d4e58ee3d248 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:29.508267 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:29.531509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efd9d0-f5bb-41f7-8c47-cfddac326e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:29.532196 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efd9d0-f5bb-41f7-8c47-cfddac326e6e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:29.532528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efd9d0-f5bb-41f7-8c47-cfddac326e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:29.532812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efd9d0-f5bb-41f7-8c47-cfddac326e6e None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:29.533550 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efd9d0-f5bb-41f7-8c47-cfddac326e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:29.534265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7926f-6507-46a0-8047-f30e472fe275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:29.534389 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efd9d0-f5bb-41f7-8c47-cfddac326e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:29.535167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7926f-6507-46a0-8047-f30e472fe275 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:29.535454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7926f-6507-46a0-8047-f30e472fe275 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:29.535736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7926f-6507-46a0-8047-f30e472fe275 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:29.537735 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efd9d0-f5bb-41f7-8c47-cfddac326e6e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:29.544989 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7926f-6507-46a0-8047-f30e472fe275 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:29.547257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7926f-6507-46a0-8047-f30e472fe275 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:29.563750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 306/619] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-93e7926f-6507-46a0-8047-f30e472fe275 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:29.565489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:29.566490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:29.567478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 314/620] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:29.572252 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efd9d0-f5bb-41f7-8c47-cfddac326e6e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:29.572547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7926f-6507-46a0-8047-f30e472fe275 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:29.712612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:29.749222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7926f-6507-46a0-8047-f30e472fe275 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:29.788362 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fbeac-9392-4e1f-9e3c-2172b3e307be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:29.789171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fbeac-9392-4e1f-9e3c-2172b3e307be None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:29.789466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fbeac-9392-4e1f-9e3c-2172b3e307be None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:29.789746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fbeac-9392-4e1f-9e3c-2172b3e307be None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:29.790595 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fbeac-9392-4e1f-9e3c-2172b3e307be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:29.790907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fbeac-9392-4e1f-9e3c-2172b3e307be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:29.792554 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fbeac-9392-4e1f-9e3c-2172b3e307be None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:29.813185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 315/621] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 14:51:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:29.820073 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fbeac-9392-4e1f-9e3c-2172b3e307be None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:29.850191 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efd9d0-f5bb-41f7-8c47-cfddac326e6e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:29.927613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fbeac-9392-4e1f-9e3c-2172b3e307be None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:29.960608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:29.973062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74b522-beb7-4974-8136-0903f63dcfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:29.974124 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74b522-beb7-4974-8136-0903f63dcfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:29.980625 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74b522-beb7-4974-8136-0903f63dcfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:29.980625 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74b522-beb7-4974-8136-0903f63dcfd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:29.986335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74b522-beb7-4974-8136-0903f63dcfd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:29.988802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74b522-beb7-4974-8136-0903f63dcfd5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:30.001564 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43814f-9150-4da1-a24c-06fb2fb9fdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:30.002303 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43814f-9150-4da1-a24c-06fb2fb9fdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.002658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43814f-9150-4da1-a24c-06fb2fb9fdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.003070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43814f-9150-4da1-a24c-06fb2fb9fdd9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-df74b522-beb7-4974-8136-0903f63dcfd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:30.008739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:30.009652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:30.010547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:30.010547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:30.010547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:30.010547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:30.010547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 316/622] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:30.015601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74b522-beb7-4974-8136-0903f63dcfd5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:30.023414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43814f-9150-4da1-a24c-06fb2fb9fdd9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:30.025165 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43814f-9150-4da1-a24c-06fb2fb9fdd9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ec43814f-9150-4da1-a24c-06fb2fb9fdd9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:30.041222 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:30.042284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:30.043433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:30.050959 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 307/623] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 14:51:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 477 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:30.052775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284fb8-8bc7-47d0-b7d1-27a12e3f950d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.053254 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284fb8-8bc7-47d0-b7d1-27a12e3f950d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.053736 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284fb8-8bc7-47d0-b7d1-27a12e3f950d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.055533 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 308/624] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:30.056462 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47284fb8-8bc7-47d0-b7d1-27a12e3f950d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:30.115347 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:30.154405 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74b522-beb7-4974-8136-0903f63dcfd5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:30.178289 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47284fb8-8bc7-47d0-b7d1-27a12e3f950d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:30.202009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae238cc-788d-4e63-94f1-86a0b1edc496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:30.202841 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae238cc-788d-4e63-94f1-86a0b1edc496 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.203130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae238cc-788d-4e63-94f1-86a0b1edc496 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.203833 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae238cc-788d-4e63-94f1-86a0b1edc496 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.204979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae238cc-788d-4e63-94f1-86a0b1edc496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:30.205314 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae238cc-788d-4e63-94f1-86a0b1edc496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:30.208125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae238cc-788d-4e63-94f1-86a0b1edc496 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:30.223009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f4d5e-927f-419c-b242-dcd79d1e885e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:30.223858 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 317/625] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:30.224332 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f4d5e-927f-419c-b242-dcd79d1e885e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.224887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f4d5e-927f-419c-b242-dcd79d1e885e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.225292 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae32d2-ecce-4cfd-98a7-02cf2d01ae72 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.225568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f4d5e-927f-419c-b242-dcd79d1e885e None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.225680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae32d2-ecce-4cfd-98a7-02cf2d01ae72 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.225849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae32d2-ecce-4cfd-98a7-02cf2d01ae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.226904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f4d5e-927f-419c-b242-dcd79d1e885e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:30.227420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f4d5e-927f-419c-b242-dcd79d1e885e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:30.229127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f4d5e-927f-419c-b242-dcd79d1e885e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:30.246725 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-adae32d2-ecce-4cfd-98a7-02cf2d01ae72 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:30.248042 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-adae32d2-ecce-4cfd-98a7-02cf2d01ae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:30.263274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 309/626] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 14:51:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:30.264610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f4d5e-927f-419c-b242-dcd79d1e885e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:30.282672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:30.344493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 318/627] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:30.345991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae32d2-ecce-4cfd-98a7-02cf2d01ae72 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:30.393036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f4d5e-927f-419c-b242-dcd79d1e885e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:30.396891 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:30.442741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f338f15-6956-4ad1-a841-97388f93ee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:30.443547 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338f15-6956-4ad1-a841-97388f93ee02 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.443835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338f15-6956-4ad1-a841-97388f93ee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.444118 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f338f15-6956-4ad1-a841-97388f93ee02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.453783 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f338f15-6956-4ad1-a841-97388f93ee02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:30.455571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f338f15-6956-4ad1-a841-97388f93ee02 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4f338f15-6956-4ad1-a841-97388f93ee02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:30.474575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:30.475531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:30.476458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:30.476458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:30.476458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:30.476458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:30.476458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:30.476458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:30.476458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:30.476458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:30.476458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 310/628] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:30.477055 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f338f15-6956-4ad1-a841-97388f93ee02 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:30.490383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae32d2-ecce-4cfd-98a7-02cf2d01ae72 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:30.544799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa1867-c8e0-4faa-898d-6fd444ed383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:30.546396 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa1867-c8e0-4faa-898d-6fd444ed383b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.546396 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa1867-c8e0-4faa-898d-6fd444ed383b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.546716 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa1867-c8e0-4faa-898d-6fd444ed383b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.584467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:30.615015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f338f15-6956-4ad1-a841-97388f93ee02 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:30.625084 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa1867-c8e0-4faa-898d-6fd444ed383b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:30.626860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa1867-c8e0-4faa-898d-6fd444ed383b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:30.652653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714dec3-49b9-4210-8179-6c6d87798aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:30.653450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714dec3-49b9-4210-8179-6c6d87798aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.653743 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714dec3-49b9-4210-8179-6c6d87798aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.654024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714dec3-49b9-4210-8179-6c6d87798aa2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.666936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714dec3-49b9-4210-8179-6c6d87798aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3714dec3-49b9-4210-8179-6c6d87798aa2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:30.677103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:30.678327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:30.679365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:30.681606 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 311/629] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:30.683225 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714dec3-49b9-4210-8179-6c6d87798aa2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:30.712203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:30.758312 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 319/630] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:30.759702 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc637007-1516-4db4-8a38-bb53aec1917b None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.760002 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc637007-1516-4db4-8a38-bb53aec1917b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.760321 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc637007-1516-4db4-8a38-bb53aec1917b None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.761583 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 320/631] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:30.762371 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc637007-1516-4db4-8a38-bb53aec1917b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:30.797810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714dec3-49b9-4210-8179-6c6d87798aa2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:30.825831 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:30.832678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083deb6-e535-40f3-9151-d1f90d1e26a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:30.838326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083deb6-e535-40f3-9151-d1f90d1e26a8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.838326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083deb6-e535-40f3-9151-d1f90d1e26a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.838326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083deb6-e535-40f3-9151-d1f90d1e26a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.838326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083deb6-e535-40f3-9151-d1f90d1e26a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:30.838326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083deb6-e535-40f3-9151-d1f90d1e26a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:30.838326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083deb6-e535-40f3-9151-d1f90d1e26a8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:30.849223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 312/632] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:30.850509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6cb91-edf4-4525-aa14-6613d81fafde None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.850809 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6cb91-edf4-4525-aa14-6613d81fafde None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.851095 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6cb91-edf4-4525-aa14-6613d81fafde None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.852620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 313/633] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:30.853338 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6cb91-edf4-4525-aa14-6613d81fafde None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:30.866910 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc637007-1516-4db4-8a38-bb53aec1917b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:30.901351 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70295d4b-3605-41ae-9daf-339c58329a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:30.902023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70295d4b-3605-41ae-9daf-339c58329a19 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.902319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70295d4b-3605-41ae-9daf-339c58329a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.902608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70295d4b-3605-41ae-9daf-339c58329a19 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.903360 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70295d4b-3605-41ae-9daf-339c58329a19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:30.903679 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70295d4b-3605-41ae-9daf-339c58329a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:30.905094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70295d4b-3605-41ae-9daf-339c58329a19 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:30.913752 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 321/634] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:30.914945 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3467b75-29d3-410d-9457-6b6c8f551dd9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.915241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3467b75-29d3-410d-9457-6b6c8f551dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.915546 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3467b75-29d3-410d-9457-6b6c8f551dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:30.931970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c3467b75-29d3-410d-9457-6b6c8f551dd9 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:30.933200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c3467b75-29d3-410d-9457-6b6c8f551dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:30.943233 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:30.959462 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6cb91-edf4-4525-aa14-6613d81fafde None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:30.994457 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2631edb-05a4-4c98-b9d5-6561c1fcf50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:30.995369 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2631edb-05a4-4c98-b9d5-6561c1fcf50d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:30.995664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2631edb-05a4-4c98-b9d5-6561c1fcf50d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:30.995943 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2631edb-05a4-4c98-b9d5-6561c1fcf50d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.001727 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 322/635] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:31.006309 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2631edb-05a4-4c98-b9d5-6561c1fcf50d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:31.007807 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3467b75-29d3-410d-9457-6b6c8f551dd9 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.008069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2631edb-05a4-4c98-b9d5-6561c1fcf50d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-f2631edb-05a4-4c98-b9d5-6561c1fcf50d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:31.026758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:31.027917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:31.028884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:31.028884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:31.028884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:31.028884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:31.028884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 314/636] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:31.029371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46a6fc-48d0-4192-964d-5ce6dd95e160 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.029659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46a6fc-48d0-4192-964d-5ce6dd95e160 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.029938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46a6fc-48d0-4192-964d-5ce6dd95e160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.047111 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1c46a6fc-48d0-4192-964d-5ce6dd95e160 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:31.048401 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1c46a6fc-48d0-4192-964d-5ce6dd95e160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:31.065164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:31.108531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3467b75-29d3-410d-9457-6b6c8f551dd9 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.118048 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 315/637] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:31.118907 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46a6fc-48d0-4192-964d-5ce6dd95e160 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.148940 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0c565-9174-4542-94e5-b498360d83a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:31.149845 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0c565-9174-4542-94e5-b498360d83a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.150138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0c565-9174-4542-94e5-b498360d83a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.150419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0c565-9174-4542-94e5-b498360d83a2 None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/bf0dd8c9980e4640aafd34946c4193a3/roles/b9c2d4c037024096bd7e864cea8b55e8` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.166587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0c565-9174-4542-94e5-b498360d83a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=bf0dd8c9980e4640aafd34946c4193a3, role_id=b9c2d4c037024096bd7e864cea8b55e8)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:31.168078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0c565-9174-4542-94e5-b498360d83a2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:31.188362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:31.222502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46a6fc-48d0-4192-964d-5ce6dd95e160 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.255900 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd40861-0596-4b8f-8d88-0ba786e48055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:31.256705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd40861-0596-4b8f-8d88-0ba786e48055 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.256989 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd40861-0596-4b8f-8d88-0ba786e48055 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.257518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd40861-0596-4b8f-8d88-0ba786e48055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.267669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd40861-0596-4b8f-8d88-0ba786e48055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:31.270249 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd40861-0596-4b8f-8d88-0ba786e48055 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-cfd40861-0596-4b8f-8d88-0ba786e48055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:31.288343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:31.289341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:31.290244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:31.290244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:31.290244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:31.290244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:31.290244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 316/638] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:31.290518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd40861-0596-4b8f-8d88-0ba786e48055 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.293530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 323/639] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:51:31 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/bf0dd8c9980e4640aafd34946c4193a3/roles/b9c2d4c037024096bd7e864cea8b55e8 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:31.294419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0c565-9174-4542-94e5-b498360d83a2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.409921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0c565-9174-4542-94e5-b498360d83a2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.418301 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd40861-0596-4b8f-8d88-0ba786e48055 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.418550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:31.438980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:31.446925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2691c8-e7f5-4381-bda3-62a80c6aa5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:31.447618 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2691c8-e7f5-4381-bda3-62a80c6aa5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.447894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2691c8-e7f5-4381-bda3-62a80c6aa5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.448176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2691c8-e7f5-4381-bda3-62a80c6aa5f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.449188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2691c8-e7f5-4381-bda3-62a80c6aa5f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:31.449691 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2691c8-e7f5-4381-bda3-62a80c6aa5f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:31.452577 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2691c8-e7f5-4381-bda3-62a80c6aa5f5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:31.467250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab3b07-af2e-4952-b1d4-fa1d7b435188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:31.468052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab3b07-af2e-4952-b1d4-fa1d7b435188 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.468446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 324/640] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:31.469289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2691c8-e7f5-4381-bda3-62a80c6aa5f5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.471983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab3b07-af2e-4952-b1d4-fa1d7b435188 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.472356 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab3b07-af2e-4952-b1d4-fa1d7b435188 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.486215 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab3b07-af2e-4952-b1d4-fa1d7b435188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7aab3b07-af2e-4952-b1d4-fa1d7b435188 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:31.495130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:31.496088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:31.496985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 317/641] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:31.497691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab3b07-af2e-4952-b1d4-fa1d7b435188 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.588460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2691c8-e7f5-4381-bda3-62a80c6aa5f5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.610760 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab3b07-af2e-4952-b1d4-fa1d7b435188 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.625135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ed40d-9428-49cc-b393-c20dd6881a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:31.625940 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ed40d-9428-49cc-b393-c20dd6881a1b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.626230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ed40d-9428-49cc-b393-c20dd6881a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.626513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ed40d-9428-49cc-b393-c20dd6881a1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.638270 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ed40d-9428-49cc-b393-c20dd6881a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-bc1ed40d-9428-49cc-b393-c20dd6881a1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:31.646652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:31.647684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 325/642] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:31.648990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ed40d-9428-49cc-b393-c20dd6881a1b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.657364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d18cea-40f9-4417-aa8a-46e866b2e9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:31.658037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d18cea-40f9-4417-aa8a-46e866b2e9de None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.658322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d18cea-40f9-4417-aa8a-46e866b2e9de None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.658634 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d18cea-40f9-4417-aa8a-46e866b2e9de None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.660064 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d18cea-40f9-4417-aa8a-46e866b2e9de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:31.660352 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d18cea-40f9-4417-aa8a-46e866b2e9de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:31.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d18cea-40f9-4417-aa8a-46e866b2e9de None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:31.678215 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 318/643] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:31.678215 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d18cea-40f9-4417-aa8a-46e866b2e9de None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.686746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:31.724631 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:31.777735 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ed40d-9428-49cc-b393-c20dd6881a1b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.792403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d18cea-40f9-4417-aa8a-46e866b2e9de None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.817405 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b05c7-2c37-4a78-855d-6df317d2e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:31.818546 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b05c7-2c37-4a78-855d-6df317d2e44e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.819085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b05c7-2c37-4a78-855d-6df317d2e44e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.819450 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b05c7-2c37-4a78-855d-6df317d2e44e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.827941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b05c7-2c37-4a78-855d-6df317d2e44e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:31.829628 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c239f-8849-4353-907d-c47b4bf4a190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:31.829772 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b05c7-2c37-4a78-855d-6df317d2e44e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:31.830297 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c239f-8849-4353-907d-c47b4bf4a190 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.831165 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c239f-8849-4353-907d-c47b4bf4a190 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.831498 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c239f-8849-4353-907d-c47b4bf4a190 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.832399 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c239f-8849-4353-907d-c47b4bf4a190 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:31.832715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c239f-8849-4353-907d-c47b4bf4a190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:31.834083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c239f-8849-4353-907d-c47b4bf4a190 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:31.847011 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 319/644] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:31.847011 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c239f-8849-4353-907d-c47b4bf4a190 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-945b05c7-2c37-4a78-855d-6df317d2e44e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:31.848188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:31.849130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 326/645] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:31.850750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b05c7-2c37-4a78-855d-6df317d2e44e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:31.947513 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:31.950671 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c239f-8849-4353-907d-c47b4bf4a190 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.962459 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:31.977743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b05c7-2c37-4a78-855d-6df317d2e44e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:31.987486 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a577cb9-d149-4e02-94e2-2f9b2138e95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:31.988668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577cb9-d149-4e02-94e2-2f9b2138e95f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:31.989015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577cb9-d149-4e02-94e2-2f9b2138e95f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:31.989301 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a577cb9-d149-4e02-94e2-2f9b2138e95f None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:31.990082 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a577cb9-d149-4e02-94e2-2f9b2138e95f None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:31.990396 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a577cb9-d149-4e02-94e2-2f9b2138e95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:31.991820 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a577cb9-d149-4e02-94e2-2f9b2138e95f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.001291 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 320/646] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:32.002186 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a577cb9-d149-4e02-94e2-2f9b2138e95f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.019241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aa167-5090-4fd3-8a2f-bcaa3a1695a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.020059 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aa167-5090-4fd3-8a2f-bcaa3a1695a5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.020380 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aa167-5090-4fd3-8a2f-bcaa3a1695a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.020661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aa167-5090-4fd3-8a2f-bcaa3a1695a5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.028091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8aa167-5090-4fd3-8a2f-bcaa3a1695a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.029801 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8aa167-5090-4fd3-8a2f-bcaa3a1695a5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1b8aa167-5090-4fd3-8a2f-bcaa3a1695a5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:32.044803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:32.045798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 327/647] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:32.051668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aa167-5090-4fd3-8a2f-bcaa3a1695a5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.119272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a577cb9-d149-4e02-94e2-2f9b2138e95f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:32.154724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aa167-5090-4fd3-8a2f-bcaa3a1695a5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:32.159158 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242607d1-e05a-4d2c-9419-4ef8d81b1696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.160118 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242607d1-e05a-4d2c-9419-4ef8d81b1696 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.160443 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242607d1-e05a-4d2c-9419-4ef8d81b1696 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.160796 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242607d1-e05a-4d2c-9419-4ef8d81b1696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.171694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242607d1-e05a-4d2c-9419-4ef8d81b1696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.173180 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242607d1-e05a-4d2c-9419-4ef8d81b1696 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-242607d1-e05a-4d2c-9419-4ef8d81b1696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:32.190538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:32.191459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:32.197157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:32.197157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:32.197157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:32.197157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:32.197447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa55d9-f089-42ba-9de8-c7434262a247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.197447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa55d9-f089-42ba-9de8-c7434262a247 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.197447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa55d9-f089-42ba-9de8-c7434262a247 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.197743 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 321/648] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:32.197852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa55d9-f089-42ba-9de8-c7434262a247 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.197938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242607d1-e05a-4d2c-9419-4ef8d81b1696 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.198022 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa55d9-f089-42ba-9de8-c7434262a247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:32.198022 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa55d9-f089-42ba-9de8-c7434262a247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.198022 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa55d9-f089-42ba-9de8-c7434262a247 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.213552 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 328/649] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:32.214442 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa55d9-f089-42ba-9de8-c7434262a247 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.218946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:32.224339 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:32.314109 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242607d1-e05a-4d2c-9419-4ef8d81b1696 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:32.320380 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa55d9-f089-42ba-9de8-c7434262a247 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:32.351244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd76a-0ce3-410b-bae4-6ae275c067ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.352067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd76a-0ce3-410b-bae4-6ae275c067ff None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.352389 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd76a-0ce3-410b-bae4-6ae275c067ff None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.352675 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd76a-0ce3-410b-bae4-6ae275c067ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.354739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca368d25-19f0-4c16-b977-eb1335857180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.355415 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca368d25-19f0-4c16-b977-eb1335857180 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.355707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca368d25-19f0-4c16-b977-eb1335857180 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.355993 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca368d25-19f0-4c16-b977-eb1335857180 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.356903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca368d25-19f0-4c16-b977-eb1335857180 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:32.357219 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca368d25-19f0-4c16-b977-eb1335857180 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.358656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca368d25-19f0-4c16-b977-eb1335857180 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.364973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cd76a-0ce3-410b-bae4-6ae275c067ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.366652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cd76a-0ce3-410b-bae4-6ae275c067ff None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.378052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 329/650] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:32.379019 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca368d25-19f0-4c16-b977-eb1335857180 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-585cd76a-0ce3-410b-bae4-6ae275c067ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:32.391362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:32.392330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:32.393529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:32.393529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:32.393529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:32.393529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:32.395374 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 322/651] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:32.396514 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd76a-0ce3-410b-bae4-6ae275c067ff None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.453830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:32.473571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:32.492372 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca368d25-19f0-4c16-b977-eb1335857180 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:32.520452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd76a-0ce3-410b-bae4-6ae275c067ff None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:32.528793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888177bb-a1fe-409b-99ce-ca7f74310cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.529464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888177bb-a1fe-409b-99ce-ca7f74310cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.529753 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888177bb-a1fe-409b-99ce-ca7f74310cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.530044 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888177bb-a1fe-409b-99ce-ca7f74310cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.530778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888177bb-a1fe-409b-99ce-ca7f74310cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:32.531088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888177bb-a1fe-409b-99ce-ca7f74310cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.532530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888177bb-a1fe-409b-99ce-ca7f74310cb5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.544689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 330/652] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:32.545524 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888177bb-a1fe-409b-99ce-ca7f74310cb5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.564525 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afb8a6-4173-4b8e-874d-8b0b4ff8770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.565899 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afb8a6-4173-4b8e-874d-8b0b4ff8770d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.566257 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afb8a6-4173-4b8e-874d-8b0b4ff8770d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.566668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afb8a6-4173-4b8e-874d-8b0b4ff8770d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.567447 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afb8a6-4173-4b8e-874d-8b0b4ff8770d None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=5368284e42d84e77a6fdb88c2e8dcd3d) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:32.567751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afb8a6-4173-4b8e-874d-8b0b4ff8770d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.569217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afb8a6-4173-4b8e-874d-8b0b4ff8770d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.580305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 323/653] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/role_assignments?user.id=5368284e42d84e77a6fdb88c2e8dcd3d&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:32.580305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afb8a6-4173-4b8e-874d-8b0b4ff8770d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.676343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888177bb-a1fe-409b-99ce-ca7f74310cb5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:32.708999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:32.720807 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90df13-6b92-465f-b140-950967c4c215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.722158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90df13-6b92-465f-b140-950967c4c215 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.722753 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90df13-6b92-465f-b140-950967c4c215 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.723143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90df13-6b92-465f-b140-950967c4c215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.741384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afb8a6-4173-4b8e-874d-8b0b4ff8770d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:32.781777 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:32.791820 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524788aa-0931-41d2-94c7-be76117ac874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.792750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524788aa-0931-41d2-94c7-be76117ac874 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.793236 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524788aa-0931-41d2-94c7-be76117ac874 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.793527 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524788aa-0931-41d2-94c7-be76117ac874 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.794757 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524788aa-0931-41d2-94c7-be76117ac874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:32.794757 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524788aa-0931-41d2-94c7-be76117ac874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.796126 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524788aa-0931-41d2-94c7-be76117ac874 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.809226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 324/654] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:32.813814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524788aa-0931-41d2-94c7-be76117ac874 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.813970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90df13-6b92-465f-b140-950967c4c215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.813970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90df13-6b92-465f-b140-950967c4c215 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:32.939432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524788aa-0931-41d2-94c7-be76117ac874 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:32.941639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 331/655] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:32.942595 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90df13-6b92-465f-b140-950967c4c215 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:32.975074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:32.976199 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7099909-e0c2-4de7-b3f8-9ecb27ca958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:32.977058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7099909-e0c2-4de7-b3f8-9ecb27ca958d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:32.977345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7099909-e0c2-4de7-b3f8-9ecb27ca958d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:32.977627 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7099909-e0c2-4de7-b3f8-9ecb27ca958d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:32.985731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7099909-e0c2-4de7-b3f8-9ecb27ca958d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:32.987887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7099909-e0c2-4de7-b3f8-9ecb27ca958d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d7099909-e0c2-4de7-b3f8-9ecb27ca958d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:33.005002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:33.006023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:33.006964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 325/656] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:33.007832 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7099909-e0c2-4de7-b3f8-9ecb27ca958d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:33.060592 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90df13-6b92-465f-b140-950967c4c215 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:33.061628 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:33.099087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fafdc-d848-4486-bd49-a27f3d18bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:33.099995 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fafdc-d848-4486-bd49-a27f3d18bf28 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:33.100310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fafdc-d848-4486-bd49-a27f3d18bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:33.100705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fafdc-d848-4486-bd49-a27f3d18bf28 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:33.108754 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fafdc-d848-4486-bd49-a27f3d18bf28 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:33.110421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fafdc-d848-4486-bd49-a27f3d18bf28 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:33.116724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7099909-e0c2-4de7-b3f8-9ecb27ca958d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-b48fafdc-d848-4486-bd49-a27f3d18bf28 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:33.126661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:33.129708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 332/657] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:33.131899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fafdc-d848-4486-bd49-a27f3d18bf28 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:33.153626 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215fa79-11ec-43a0-a401-3759e5a3df9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:33.154431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215fa79-11ec-43a0-a401-3759e5a3df9d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:33.154722 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215fa79-11ec-43a0-a401-3759e5a3df9d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:33.155007 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215fa79-11ec-43a0-a401-3759e5a3df9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:33.166374 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215fa79-11ec-43a0-a401-3759e5a3df9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:33.167842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215fa79-11ec-43a0-a401-3759e5a3df9d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c215fa79-11ec-43a0-a401-3759e5a3df9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:33.188593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:33.191475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:33.193598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:33.193598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:33.193598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:33.193598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:33.196643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 326/658] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:33.196643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215fa79-11ec-43a0-a401-3759e5a3df9d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:33.227804 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:33.246942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fafdc-d848-4486-bd49-a27f3d18bf28 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:33.285384 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35436a4f-3830-4a75-8c7e-bd44860de2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:33.286166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35436a4f-3830-4a75-8c7e-bd44860de2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:33.286499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35436a4f-3830-4a75-8c7e-bd44860de2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:33.286951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35436a4f-3830-4a75-8c7e-bd44860de2b5 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:33.287814 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35436a4f-3830-4a75-8c7e-bd44860de2b5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:33.288366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35436a4f-3830-4a75-8c7e-bd44860de2b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:33.289956 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35436a4f-3830-4a75-8c7e-bd44860de2b5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:33.309399 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215fa79-11ec-43a0-a401-3759e5a3df9d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:33.310220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 333/659] 10.222.0.88 () {56 vars in 1139 bytes} [Thu Jun 16 14:51:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:33.311149 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35436a4f-3830-4a75-8c7e-bd44860de2b5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:33.320253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:33.355075 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa86b92-872b-4931-ab02-588f484ffeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:33.355860 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa86b92-872b-4931-ab02-588f484ffeae None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:33.356212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa86b92-872b-4931-ab02-588f484ffeae None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:33.356602 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa86b92-872b-4931-ab02-588f484ffeae None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:33.357801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa86b92-872b-4931-ab02-588f484ffeae None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:33.358179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa86b92-872b-4931-ab02-588f484ffeae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:33.359818 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa86b92-872b-4931-ab02-588f484ffeae None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:33.387243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 327/660] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 14:51:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:33.388372 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa86b92-872b-4931-ab02-588f484ffeae None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:33.427316 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35436a4f-3830-4a75-8c7e-bd44860de2b5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:33.468568 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a3672-7d19-44a4-b982-627e2c878477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:33.469291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a3672-7d19-44a4-b982-627e2c878477 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:33.469583 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a3672-7d19-44a4-b982-627e2c878477 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:33.469867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a3672-7d19-44a4-b982-627e2c878477 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:33.470604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a3672-7d19-44a4-b982-627e2c878477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:33.470948 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a3672-7d19-44a4-b982-627e2c878477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:33.472411 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a3672-7d19-44a4-b982-627e2c878477 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:33.490985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 334/661] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:33.491963 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a3672-7d19-44a4-b982-627e2c878477 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:33.494997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:33.504911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa86b92-872b-4931-ab02-588f484ffeae None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:33.547915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d1981-2ca6-4bff-93d2-f81f0a099e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:33.548606 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d1981-2ca6-4bff-93d2-f81f0a099e53 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:33.548896 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d1981-2ca6-4bff-93d2-f81f0a099e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:33.549351 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d1981-2ca6-4bff-93d2-f81f0a099e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:33.582927 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:33.616508 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a3672-7d19-44a4-b982-627e2c878477 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:33.622267 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d1981-2ca6-4bff-93d2-f81f0a099e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:33.623749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d1981-2ca6-4bff-93d2-f81f0a099e53 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:33.652037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12366e59-1591-4d91-beff-007b03fc3d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:33.652724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12366e59-1591-4d91-beff-007b03fc3d25 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:33.653031 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12366e59-1591-4d91-beff-007b03fc3d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:33.653298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12366e59-1591-4d91-beff-007b03fc3d25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:33.733538 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12366e59-1591-4d91-beff-007b03fc3d25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:33.735834 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12366e59-1591-4d91-beff-007b03fc3d25 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:33.741479 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 328/662] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:33.741479 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d1981-2ca6-4bff-93d2-f81f0a099e53 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:33.744430 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:33.806654 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:33.848803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d1981-2ca6-4bff-93d2-f81f0a099e53 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:33.855765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 335/663] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:51:33.857310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea2324-0131-4e8c-a8a9-7da71cba0626 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:33.857602 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea2324-0131-4e8c-a8a9-7da71cba0626 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:33.857883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea2324-0131-4e8c-a8a9-7da71cba0626 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:33.862958 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 336/664] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:33.862958 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea2324-0131-4e8c-a8a9-7da71cba0626 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:33.892891 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07eb65-09a7-43ed-b936-7480e7042282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:33.893721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07eb65-09a7-43ed-b936-7480e7042282 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:33.894007 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07eb65-09a7-43ed-b936-7480e7042282 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:33.894289 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07eb65-09a7-43ed-b936-7480e7042282 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:33.901750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07eb65-09a7-43ed-b936-7480e7042282 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:33.903208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07eb65-09a7-43ed-b936-7480e7042282 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4f07eb65-09a7-43ed-b936-7480e7042282 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:33.918061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:33.919018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:33.919971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:33.919971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:33.919971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:33.919971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:33.919971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:33.919971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:33.919971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:33.919971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:33.919971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 329/665] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:33.920513 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07eb65-09a7-43ed-b936-7480e7042282 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:33.974419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea2324-0131-4e8c-a8a9-7da71cba0626 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:33.975992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:34.016713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d80ae-f123-45a9-be7e-aaf86ecc381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.017424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d80ae-f123-45a9-be7e-aaf86ecc381a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.017890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d80ae-f123-45a9-be7e-aaf86ecc381a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.018697 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d80ae-f123-45a9-be7e-aaf86ecc381a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.029131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d80ae-f123-45a9-be7e-aaf86ecc381a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:34.029423 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07eb65-09a7-43ed-b936-7480e7042282 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:34.030589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d80ae-f123-45a9-be7e-aaf86ecc381a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-315d80ae-f123-45a9-be7e-aaf86ecc381a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:34.048777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:34.049806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:34.050747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:34.050747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:34.050747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:34.050747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:34.050747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 337/666] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:34.051614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ed00-052e-4885-80a9-c0aac0a18275 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.051907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ed00-052e-4885-80a9-c0aac0a18275 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.052187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ed00-052e-4885-80a9-c0aac0a18275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.061179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:34.069173 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-82e6ed00-052e-4885-80a9-c0aac0a18275 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:34.070351 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-82e6ed00-052e-4885-80a9-c0aac0a18275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:34.078163 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532602a-ca52-43e9-83f1-9d75a7db2ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.078965 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532602a-ca52-43e9-83f1-9d75a7db2ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.079250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532602a-ca52-43e9-83f1-9d75a7db2ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.079525 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532602a-ca52-43e9-83f1-9d75a7db2ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.092130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532602a-ca52-43e9-83f1-9d75a7db2ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:34.093620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532602a-ca52-43e9-83f1-9d75a7db2ed0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d532602a-ca52-43e9-83f1-9d75a7db2ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:34.112715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:34.113822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:34.114744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:34.114744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:34.114744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:34.114744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:34.114744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 330/667] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:34.115647 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532602a-ca52-43e9-83f1-9d75a7db2ed0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.146815 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 338/668] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:34.147662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ed00-052e-4885-80a9-c0aac0a18275 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.242581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532602a-ca52-43e9-83f1-9d75a7db2ed0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:34.243501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:34.260516 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ed00-052e-4885-80a9-c0aac0a18275 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:34.288617 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f3abf-f443-4938-af32-a3c74cde1b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.289580 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f3abf-f443-4938-af32-a3c74cde1b66 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.289865 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f3abf-f443-4938-af32-a3c74cde1b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.290148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f3abf-f443-4938-af32-a3c74cde1b66 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.290940 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f3abf-f443-4938-af32-a3c74cde1b66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:34.291252 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f3abf-f443-4938-af32-a3c74cde1b66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:34.292713 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f3abf-f443-4938-af32-a3c74cde1b66 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:34.300492 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58a6c7-a31e-4c06-898c-a5de5d66587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.301960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58a6c7-a31e-4c06-898c-a5de5d66587c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.302427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58a6c7-a31e-4c06-898c-a5de5d66587c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.303341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58a6c7-a31e-4c06-898c-a5de5d66587c None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.304383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58a6c7-a31e-4c06-898c-a5de5d66587c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:34.304871 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58a6c7-a31e-4c06-898c-a5de5d66587c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:34.306012 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 331/669] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:34.306680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58a6c7-a31e-4c06-898c-a5de5d66587c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:34.306852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f3abf-f443-4938-af32-a3c74cde1b66 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.325282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 339/670] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:34.326617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58a6c7-a31e-4c06-898c-a5de5d66587c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.357117 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:34.442222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58a6c7-a31e-4c06-898c-a5de5d66587c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:34.446721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f3abf-f443-4938-af32-a3c74cde1b66 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:34.481856 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57719219-c079-42de-91b2-b2d33608b79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.482747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57719219-c079-42de-91b2-b2d33608b79b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.482747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57719219-c079-42de-91b2-b2d33608b79b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.483447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57719219-c079-42de-91b2-b2d33608b79b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.490458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425615c-156d-47a0-8596-cd526e275162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.491345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425615c-156d-47a0-8596-cd526e275162 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.491460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425615c-156d-47a0-8596-cd526e275162 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.491894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425615c-156d-47a0-8596-cd526e275162 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.495256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425615c-156d-47a0-8596-cd526e275162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:34.495256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425615c-156d-47a0-8596-cd526e275162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:34.495256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425615c-156d-47a0-8596-cd526e275162 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:34.504264 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57719219-c079-42de-91b2-b2d33608b79b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:34.509265 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 332/671] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:34.510109 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425615c-156d-47a0-8596-cd526e275162 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-57719219-c079-42de-91b2-b2d33608b79b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:34.515732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:34.516793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:34.517703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:34.522117 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 340/672] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:34.522897 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57719219-c079-42de-91b2-b2d33608b79b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.529145 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:34.622175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:34.633128 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425615c-156d-47a0-8596-cd526e275162 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:34.636035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57719219-c079-42de-91b2-b2d33608b79b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:34.670602 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4876267-4e2e-44cc-8a6f-f45b757d2a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.671256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4876267-4e2e-44cc-8a6f-f45b757d2a55 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.671543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4876267-4e2e-44cc-8a6f-f45b757d2a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.671820 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4876267-4e2e-44cc-8a6f-f45b757d2a55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.672479 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a431c-c9e5-4784-b317-60afb0163dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.672686 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4876267-4e2e-44cc-8a6f-f45b757d2a55 None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=e7ffcec55ac14f5bb8c3af6be2d4c7c6) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:34.672997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4876267-4e2e-44cc-8a6f-f45b757d2a55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:34.673461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a431c-c9e5-4784-b317-60afb0163dff None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.673813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a431c-c9e5-4784-b317-60afb0163dff None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.675066 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a431c-c9e5-4784-b317-60afb0163dff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.675233 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4876267-4e2e-44cc-8a6f-f45b757d2a55 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:34.692664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 333/673] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/role_assignments?user.id=e7ffcec55ac14f5bb8c3af6be2d4c7c6&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:34.692913 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a431c-c9e5-4784-b317-60afb0163dff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:34.693004 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4876267-4e2e-44cc-8a6f-f45b757d2a55 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.693079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a431c-c9e5-4784-b317-60afb0163dff None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-453a431c-c9e5-4784-b317-60afb0163dff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:34.710493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:34.711684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:34.712838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:34.712838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:34.712838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:34.712838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:34.712838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:34.712838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:34.712838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:34.712838 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:34.714215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 341/674] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:34.715162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a431c-c9e5-4784-b317-60afb0163dff None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.794741 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:34.840602 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4876267-4e2e-44cc-8a6f-f45b757d2a55 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:34.853128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a431c-c9e5-4784-b317-60afb0163dff None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:34.894789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b546160-3178-4174-b564-4a206d3e0ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.895610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b546160-3178-4174-b564-4a206d3e0ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.895898 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b546160-3178-4174-b564-4a206d3e0ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.896176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b546160-3178-4174-b564-4a206d3e0ea0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.896375 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea98750-f9ec-4cba-ad65-1bb411cb5f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:34.896953 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea98750-f9ec-4cba-ad65-1bb411cb5f38 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:34.898559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea98750-f9ec-4cba-ad65-1bb411cb5f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:34.898559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea98750-f9ec-4cba-ad65-1bb411cb5f38 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:34.898559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea98750-f9ec-4cba-ad65-1bb411cb5f38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:34.898801 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea98750-f9ec-4cba-ad65-1bb411cb5f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:34.900000 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea98750-f9ec-4cba-ad65-1bb411cb5f38 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:34.904600 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b546160-3178-4174-b564-4a206d3e0ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:34.906196 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b546160-3178-4174-b564-4a206d3e0ea0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:34.910143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 342/675] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:34.911298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea98750-f9ec-4cba-ad65-1bb411cb5f38 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-6b546160-3178-4174-b564-4a206d3e0ea0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:34.927326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:34.928365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:34.929711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:34.929711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:34.929711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:34.929711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:34.929711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:34.929711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:34.929711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:34.929711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:34.929711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 334/676] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:34.930221 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b546160-3178-4174-b564-4a206d3e0ea0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:34.955963 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:35.061783 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b546160-3178-4174-b564-4a206d3e0ea0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:35.063829 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea98750-f9ec-4cba-ad65-1bb411cb5f38 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:35.093901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:35.103880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bf30d-5a2f-4b41-867d-f3a08a6f96ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:35.106074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bf30d-5a2f-4b41-867d-f3a08a6f96ad None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.106074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bf30d-5a2f-4b41-867d-f3a08a6f96ad None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.106265 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fc7b3-94b5-48ac-8b38-789ac6ba430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:35.106377 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bf30d-5a2f-4b41-867d-f3a08a6f96ad None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.106377 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16bf30d-5a2f-4b41-867d-f3a08a6f96ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:35.106509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fc7b3-94b5-48ac-8b38-789ac6ba430b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.106580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16bf30d-5a2f-4b41-867d-f3a08a6f96ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:35.106649 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fc7b3-94b5-48ac-8b38-789ac6ba430b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.106910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fc7b3-94b5-48ac-8b38-789ac6ba430b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.107925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16bf30d-5a2f-4b41-867d-f3a08a6f96ad None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:35.118936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fc7b3-94b5-48ac-8b38-789ac6ba430b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:35.120437 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fc7b3-94b5-48ac-8b38-789ac6ba430b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:35.124651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 343/677] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:35.125958 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bf30d-5a2f-4b41-867d-f3a08a6f96ad None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c56fc7b3-94b5-48ac-8b38-789ac6ba430b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:35.143509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:35.144666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:35.145828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:35.145828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:35.145828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:35.145828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:35.145828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 335/678] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:35.146371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b4482-95ed-4c13-a8b9-ea58bdfae91d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.146659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b4482-95ed-4c13-a8b9-ea58bdfae91d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.146941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b4482-95ed-4c13-a8b9-ea58bdfae91d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.148413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 336/679] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:35.152970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b4482-95ed-4c13-a8b9-ea58bdfae91d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:35.230571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:35.259304 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bf30d-5a2f-4b41-867d-f3a08a6f96ad None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:35.272700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b4482-95ed-4c13-a8b9-ea58bdfae91d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:35.311202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613c717-ae31-46e6-9353-21d134b6156d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:35.311869 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c717-ae31-46e6-9353-21d134b6156d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.312153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c717-ae31-46e6-9353-21d134b6156d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.312467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c717-ae31-46e6-9353-21d134b6156d None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.313257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613c717-ae31-46e6-9353-21d134b6156d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:35.313569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613c717-ae31-46e6-9353-21d134b6156d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:35.314682 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532f1a4-0182-4764-af64-c194e084dc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:35.315989 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532f1a4-0182-4764-af64-c194e084dc4f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.316296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613c717-ae31-46e6-9353-21d134b6156d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:35.316653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532f1a4-0182-4764-af64-c194e084dc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.317080 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532f1a4-0182-4764-af64-c194e084dc4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.318261 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532f1a4-0182-4764-af64-c194e084dc4f None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=58a2b345fe914d2cb97756e6fdf30010) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:35.318734 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532f1a4-0182-4764-af64-c194e084dc4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:35.320698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532f1a4-0182-4764-af64-c194e084dc4f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:35.327932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 337/680] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:35 2022] GET /identity/v3/role_assignments?user.id=58a2b345fe914d2cb97756e6fdf30010&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:35.329011 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532f1a4-0182-4764-af64-c194e084dc4f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:35.329308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 344/681] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:35.331331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e11c6e-8060-4336-8a4c-9ee2a7ac4fc8 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.331622 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e11c6e-8060-4336-8a4c-9ee2a7ac4fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.331901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e11c6e-8060-4336-8a4c-9ee2a7ac4fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.353478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c6e11c6e-8060-4336-8a4c-9ee2a7ac4fc8 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:35.355640 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c6e11c6e-8060-4336-8a4c-9ee2a7ac4fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:35.421585 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:35.460257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 345/682] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:35.460257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e11c6e-8060-4336-8a4c-9ee2a7ac4fc8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:35.468052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532f1a4-0182-4764-af64-c194e084dc4f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:35.525871 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:35.532986 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e738ac-76d4-4f5a-9611-5375d0b4b2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:35.545290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e738ac-76d4-4f5a-9611-5375d0b4b2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.545290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e738ac-76d4-4f5a-9611-5375d0b4b2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.545290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e738ac-76d4-4f5a-9611-5375d0b4b2f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.545290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e738ac-76d4-4f5a-9611-5375d0b4b2f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:35.545290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e738ac-76d4-4f5a-9611-5375d0b4b2f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:35.545290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e738ac-76d4-4f5a-9611-5375d0b4b2f5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:35.552588 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 338/683] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:35.553812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e738ac-76d4-4f5a-9611-5375d0b4b2f5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:35.622366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e11c6e-8060-4336-8a4c-9ee2a7ac4fc8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:35.674141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c30522-51b9-4474-b29e-c998be18fbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:35.674879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c30522-51b9-4474-b29e-c998be18fbfe None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.675182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c30522-51b9-4474-b29e-c998be18fbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.675471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c30522-51b9-4474-b29e-c998be18fbfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.676851 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c30522-51b9-4474-b29e-c998be18fbfe None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=b39d53d1def347b6b36067e3e93aeb98) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:35.677174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c30522-51b9-4474-b29e-c998be18fbfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:35.677732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e738ac-76d4-4f5a-9611-5375d0b4b2f5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:35.678710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c30522-51b9-4474-b29e-c998be18fbfe None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:35.692402 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 346/684] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:35 2022] GET /identity/v3/role_assignments?user.id=b39d53d1def347b6b36067e3e93aeb98&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:35.693734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c30522-51b9-4474-b29e-c998be18fbfe None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:35.741918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcfc74-982c-401b-81a1-3fb44bfe2e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:35.742741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcfc74-982c-401b-81a1-3fb44bfe2e3a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.743036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcfc74-982c-401b-81a1-3fb44bfe2e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.743325 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcfc74-982c-401b-81a1-3fb44bfe2e3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.770296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:35.770932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcfc74-982c-401b-81a1-3fb44bfe2e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1fdcfc74-982c-401b-81a1-3fb44bfe2e3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:35.789616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:35.790640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:35.791503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:35.797045 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 339/685] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:35.803309 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcfc74-982c-401b-81a1-3fb44bfe2e3a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:35.864911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c30522-51b9-4474-b29e-c998be18fbfe None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:35.906050 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0ea5c-08d5-44f4-8eb4-cc41b3de09fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:35.906868 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0ea5c-08d5-44f4-8eb4-cc41b3de09fa None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:35.907155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0ea5c-08d5-44f4-8eb4-cc41b3de09fa None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:35.907436 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0ea5c-08d5-44f4-8eb4-cc41b3de09fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:35.919766 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:35.926837 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0ea5c-08d5-44f4-8eb4-cc41b3de09fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:35.928598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0ea5c-08d5-44f4-8eb4-cc41b3de09fa None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-47b0ea5c-08d5-44f4-8eb4-cc41b3de09fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:35.946912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:35.947934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:35.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:35.951633 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 347/686] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:35.957434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0ea5c-08d5-44f4-8eb4-cc41b3de09fa None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:35.985169 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcfc74-982c-401b-81a1-3fb44bfe2e3a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:36.024147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e56c0-fa5c-474f-8c5c-72fced0e0724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:36.024896 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e56c0-fa5c-474f-8c5c-72fced0e0724 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:36.025190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e56c0-fa5c-474f-8c5c-72fced0e0724 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:36.025476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e56c0-fa5c-474f-8c5c-72fced0e0724 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:36.026236 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e56c0-fa5c-474f-8c5c-72fced0e0724 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:36.026549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e56c0-fa5c-474f-8c5c-72fced0e0724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:36.027925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e56c0-fa5c-474f-8c5c-72fced0e0724 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:36.037948 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 340/687] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:36.043649 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e56c0-fa5c-474f-8c5c-72fced0e0724 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:36.081150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:36.084368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0ea5c-08d5-44f4-8eb4-cc41b3de09fa None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:36.122513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c98d7-a826-47d1-851e-d0b6cc33a74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:36.123774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c98d7-a826-47d1-851e-d0b6cc33a74f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:36.124211 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c98d7-a826-47d1-851e-d0b6cc33a74f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:36.124607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c98d7-a826-47d1-851e-d0b6cc33a74f None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:36.125631 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c98d7-a826-47d1-851e-d0b6cc33a74f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:36.126020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c98d7-a826-47d1-851e-d0b6cc33a74f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:36.128139 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c98d7-a826-47d1-851e-d0b6cc33a74f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:36.156373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 348/688] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 14:51:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:36.158713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c98d7-a826-47d1-851e-d0b6cc33a74f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:36.164100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e56c0-fa5c-474f-8c5c-72fced0e0724 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:36.182671 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:36.202438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9b38e-cc4f-43a2-a22d-ed0564b689bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:36.203566 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9b38e-cc4f-43a2-a22d-ed0564b689bc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:36.204199 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9b38e-cc4f-43a2-a22d-ed0564b689bc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:36.204830 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9b38e-cc4f-43a2-a22d-ed0564b689bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:36.218670 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9b38e-cc4f-43a2-a22d-ed0564b689bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:36.221543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9b38e-cc4f-43a2-a22d-ed0564b689bc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d6b9b38e-cc4f-43a2-a22d-ed0564b689bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:36.241208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:36.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:36.243089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:36.243089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:36.243089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:36.243089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:36.245940 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 341/689] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:36.251620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9b38e-cc4f-43a2-a22d-ed0564b689bc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:36.367983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9b38e-cc4f-43a2-a22d-ed0564b689bc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:36.406748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421508b8-3d3f-4a48-bf31-f731c8e65427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:36.407758 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421508b8-3d3f-4a48-bf31-f731c8e65427 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:36.408988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421508b8-3d3f-4a48-bf31-f731c8e65427 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:36.409603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421508b8-3d3f-4a48-bf31-f731c8e65427 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:36.410520 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421508b8-3d3f-4a48-bf31-f731c8e65427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:36.411024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421508b8-3d3f-4a48-bf31-f731c8e65427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:36.412814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421508b8-3d3f-4a48-bf31-f731c8e65427 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:36.427564 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 342/690] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:36.433425 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421508b8-3d3f-4a48-bf31-f731c8e65427 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:36.476686 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:36.535182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c98d7-a826-47d1-851e-d0b6cc33a74f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:36.544366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421508b8-3d3f-4a48-bf31-f731c8e65427 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:36.585598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e03808-6c93-4ebb-af9c-1b7e2fbad3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:36.586797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e03808-6c93-4ebb-af9c-1b7e2fbad3bb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:36.587706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e03808-6c93-4ebb-af9c-1b7e2fbad3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:36.588195 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e03808-6c93-4ebb-af9c-1b7e2fbad3bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:36.597731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e03808-6c93-4ebb-af9c-1b7e2fbad3bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:36.599700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e03808-6c93-4ebb-af9c-1b7e2fbad3bb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-f8e03808-6c93-4ebb-af9c-1b7e2fbad3bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:36.616625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:36.617643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:36.618661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:36.621725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 343/691] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 14:51:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:36.627741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e03808-6c93-4ebb-af9c-1b7e2fbad3bb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:36.672966 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b861a-e140-4f69-8728-6b166899e3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:36.674426 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b861a-e140-4f69-8728-6b166899e3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:36.674744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b861a-e140-4f69-8728-6b166899e3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:36.675135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b861a-e140-4f69-8728-6b166899e3c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:36.741607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e03808-6c93-4ebb-af9c-1b7e2fbad3bb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:36.744923 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:36.758300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:36.781219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578019c7-4eee-49b3-840b-ca851c2de52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:36.782204 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578019c7-4eee-49b3-840b-ca851c2de52f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:36.782694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578019c7-4eee-49b3-840b-ca851c2de52f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:36.783277 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578019c7-4eee-49b3-840b-ca851c2de52f None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:36.796709 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578019c7-4eee-49b3-840b-ca851c2de52f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:36.797291 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578019c7-4eee-49b3-840b-ca851c2de52f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:36.799095 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578019c7-4eee-49b3-840b-ca851c2de52f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:36.819259 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 344/692] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 14:51:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:36.825785 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578019c7-4eee-49b3-840b-ca851c2de52f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:36.835047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b861a-e140-4f69-8728-6b166899e3c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:36.836649 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b861a-e140-4f69-8728-6b166899e3c5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:36.931536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578019c7-4eee-49b3-840b-ca851c2de52f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:36.968431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0612507-498b-4dfa-8331-369a4c509695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:36.969526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0612507-498b-4dfa-8331-369a4c509695 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:36.969997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0612507-498b-4dfa-8331-369a4c509695 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:36.970456 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0612507-498b-4dfa-8331-369a4c509695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:36.988579 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:37.046014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0612507-498b-4dfa-8331-369a4c509695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:37.047763 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0612507-498b-4dfa-8331-369a4c509695 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:37.054552 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 349/693] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 896 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:51:37.059868 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b861a-e140-4f69-8728-6b166899e3c5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:37.153846 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 345/694] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:51:37.160429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0612507-498b-4dfa-8331-369a4c509695 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:37.163097 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:37.211265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b861a-e140-4f69-8728-6b166899e3c5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:37.262861 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979948-3af7-47d3-85df-19addd061ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:37.263711 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979948-3af7-47d3-85df-19addd061ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.264022 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979948-3af7-47d3-85df-19addd061ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.264627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979948-3af7-47d3-85df-19addd061ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:37.264734 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:37.286241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21979948-3af7-47d3-85df-19addd061ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:37.287924 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21979948-3af7-47d3-85df-19addd061ec4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-21979948-3af7-47d3-85df-19addd061ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:37.311141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:37.312090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:37.313025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:37.313025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:37.313025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:37.313025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:37.313025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 350/695] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:37.318770 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979948-3af7-47d3-85df-19addd061ec4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:37.353446 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0612507-498b-4dfa-8331-369a4c509695 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:37.427587 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d130bad-c1a3-4042-ada0-6ffc6c4816ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:37.428445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d130bad-c1a3-4042-ada0-6ffc6c4816ad None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.428771 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d130bad-c1a3-4042-ada0-6ffc6c4816ad None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.429069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d130bad-c1a3-4042-ada0-6ffc6c4816ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:37.451879 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d130bad-c1a3-4042-ada0-6ffc6c4816ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:37.453398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d130bad-c1a3-4042-ada0-6ffc6c4816ad None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-9d130bad-c1a3-4042-ada0-6ffc6c4816ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:37.495811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:37.496803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:37.497718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:37.497718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:37.497718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:37.497718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:37.502018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 346/696] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:37.504912 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d130bad-c1a3-4042-ada0-6ffc6c4816ad None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:37.527598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979948-3af7-47d3-85df-19addd061ec4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:37.547986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:37.585561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d5240-e2e0-4633-b79d-d58ad75093b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:37.586141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5240-e2e0-4633-b79d-d58ad75093b2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.586141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5240-e2e0-4633-b79d-d58ad75093b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.586320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5240-e2e0-4633-b79d-d58ad75093b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:37.587047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d5240-e2e0-4633-b79d-d58ad75093b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:37.587269 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d5240-e2e0-4633-b79d-d58ad75093b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:37.588681 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d5240-e2e0-4633-b79d-d58ad75093b2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:37.603125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 351/697] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:37.604587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c062ae-e2fd-4b77-874b-3c7d01b58a8b None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.604886 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c062ae-e2fd-4b77-874b-3c7d01b58a8b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.605223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c062ae-e2fd-4b77-874b-3c7d01b58a8b None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:37.610517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 352/698] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:37.610517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c062ae-e2fd-4b77-874b-3c7d01b58a8b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:37.652625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:37.659861 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d130bad-c1a3-4042-ada0-6ffc6c4816ad None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:37.707058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931afc9b-6370-4383-b16b-f78f257c446a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:37.708501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931afc9b-6370-4383-b16b-f78f257c446a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.709215 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931afc9b-6370-4383-b16b-f78f257c446a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.709998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931afc9b-6370-4383-b16b-f78f257c446a None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:37.711489 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931afc9b-6370-4383-b16b-f78f257c446a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:37.711990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931afc9b-6370-4383-b16b-f78f257c446a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:37.713812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931afc9b-6370-4383-b16b-f78f257c446a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:37.727561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c062ae-e2fd-4b77-874b-3c7d01b58a8b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:37.740074 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 347/699] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:37.743204 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d649f13-9698-447a-af41-88c2913c5844 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.744254 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d649f13-9698-447a-af41-88c2913c5844 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.745068 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d649f13-9698-447a-af41-88c2913c5844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:37.766723 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-4d649f13-9698-447a-af41-88c2913c5844 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:37.767758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bcd9b-e750-444d-9830-4c77c83e260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:37.768720 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bcd9b-e750-444d-9830-4c77c83e260a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.769049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-4d649f13-9698-447a-af41-88c2913c5844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:37.769167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bcd9b-e750-444d-9830-4c77c83e260a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.769560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bcd9b-e750-444d-9830-4c77c83e260a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:37.777475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bcd9b-e750-444d-9830-4c77c83e260a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:37.779262 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bcd9b-e750-444d-9830-4c77c83e260a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1b6bcd9b-e750-444d-9830-4c77c83e260a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:37.795202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:37.796343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:37.797560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:37.797560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:37.797560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:37.797560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:37.797560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:37.797560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:37.797560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:37.797560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:37.797560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 353/700] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:37.798065 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bcd9b-e750-444d-9830-4c77c83e260a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:37.831339 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:37.855648 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 348/701] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:37.856934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18fba0-fb0d-47c0-b64b-29b6acd0019c None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.857236 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18fba0-fb0d-47c0-b64b-29b6acd0019c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.857523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18fba0-fb0d-47c0-b64b-29b6acd0019c None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:37.859058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 349/702] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:37.859771 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18fba0-fb0d-47c0-b64b-29b6acd0019c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:37.916481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bcd9b-e750-444d-9830-4c77c83e260a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:37.955670 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a8d47-6ae2-4e31-b59e-e8580c85f811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:37.956307 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8d47-6ae2-4e31-b59e-e8580c85f811 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.956601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8d47-6ae2-4e31-b59e-e8580c85f811 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.957123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:37.957630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8d47-6ae2-4e31-b59e-e8580c85f811 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:37.965643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a8d47-6ae2-4e31-b59e-e8580c85f811 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:37.966217 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a8d47-6ae2-4e31-b59e-e8580c85f811 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:37.974248 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18fba0-fb0d-47c0-b64b-29b6acd0019c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-4a6a8d47-6ae2-4e31-b59e-e8580c85f811 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:37.981035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:37.981957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:37.982863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:37.982863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:37.982863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:37.982863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:37.982863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:37.982863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:37.982863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:37.982863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:37.982863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 354/703] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:37.984041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0ba8a-0f96-4791-bf18-671b7d70f0d6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:37.984145 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0ba8a-0f96-4791-bf18-671b7d70f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:37.984459 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0ba8a-0f96-4791-bf18-671b7d70f0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.002453 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fab0ba8a-0f96-4791-bf18-671b7d70f0d6 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:38.003647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fab0ba8a-0f96-4791-bf18-671b7d70f0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:38.014903 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27995f5-ba68-41ab-8938-8dbea6d60769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:38.015577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27995f5-ba68-41ab-8938-8dbea6d60769 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.015869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27995f5-ba68-41ab-8938-8dbea6d60769 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.016148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27995f5-ba68-41ab-8938-8dbea6d60769 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.016981 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27995f5-ba68-41ab-8938-8dbea6d60769 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:38.017296 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27995f5-ba68-41ab-8938-8dbea6d60769 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:38.019583 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27995f5-ba68-41ab-8938-8dbea6d60769 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:38.031426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 350/704] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:38.032267 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27995f5-ba68-41ab-8938-8dbea6d60769 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:38.089167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 355/705] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:38.090457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0ba8a-0f96-4791-bf18-671b7d70f0d6 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:38.126120 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:38.177270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27995f5-ba68-41ab-8938-8dbea6d60769 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:38.225325 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f6b6e-1213-4cd8-8165-6dadfef026f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:38.225667 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0ba8a-0f96-4791-bf18-671b7d70f0d6 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:38.226940 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f6b6e-1213-4cd8-8165-6dadfef026f2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.227546 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f6b6e-1213-4cd8-8165-6dadfef026f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.228301 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f6b6e-1213-4cd8-8165-6dadfef026f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.243186 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f6b6e-1213-4cd8-8165-6dadfef026f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:38.244980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f6b6e-1213-4cd8-8165-6dadfef026f2 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:38.254196 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-950f6b6e-1213-4cd8-8165-6dadfef026f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:38.265933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:38.266946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:38.267925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:38.267925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:38.267925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:38.267925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:38.272807 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 351/706] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:38.273595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f6b6e-1213-4cd8-8165-6dadfef026f2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:38.275379 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9d14-4410-4b1f-b120-2d1e3a39a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:38.275379 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9d14-4410-4b1f-b120-2d1e3a39a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.275848 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9d14-4410-4b1f-b120-2d1e3a39a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.276126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9d14-4410-4b1f-b120-2d1e3a39a7d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.276965 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b9d14-4410-4b1f-b120-2d1e3a39a7d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:38.279542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b9d14-4410-4b1f-b120-2d1e3a39a7d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:38.279542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b9d14-4410-4b1f-b120-2d1e3a39a7d3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:38.293544 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 356/707] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:38.294460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9d14-4410-4b1f-b120-2d1e3a39a7d3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:38.406935 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:38.426998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f6b6e-1213-4cd8-8165-6dadfef026f2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:38.429150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9d14-4410-4b1f-b120-2d1e3a39a7d3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:38.462343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe690c2-07b8-4568-9db7-803682c4c7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:38.462997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe690c2-07b8-4568-9db7-803682c4c7bd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.463290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe690c2-07b8-4568-9db7-803682c4c7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.463571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe690c2-07b8-4568-9db7-803682c4c7bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.463669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d7ad8-9832-4ea8-aa71-cad176062245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:38.464469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d7ad8-9832-4ea8-aa71-cad176062245 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.464582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe690c2-07b8-4568-9db7-803682c4c7bd None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=bf0dd8c9980e4640aafd34946c4193a3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:38.464690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d7ad8-9832-4ea8-aa71-cad176062245 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.464779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe690c2-07b8-4568-9db7-803682c4c7bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:38.464955 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d7ad8-9832-4ea8-aa71-cad176062245 None admin] PATH_INFO: `/v3/services/network` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.465602 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d7ad8-9832-4ea8-aa71-cad176062245 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:38.466181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe690c2-07b8-4568-9db7-803682c4c7bd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:38.466960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d7ad8-9832-4ea8-aa71-cad176062245 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:38.473597 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 352/708] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:38 2022] GET /identity/v3/role_assignments?user.id=bf0dd8c9980e4640aafd34946c4193a3&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:38.474095 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe690c2-07b8-4568-9db7-803682c4c7bd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-500d7ad8-9832-4ea8-aa71-cad176062245 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:38.481977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:38.483326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:38.484322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 357/709] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:38 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:38.485203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d7ad8-9832-4ea8-aa71-cad176062245 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:38.558310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:38.610810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe690c2-07b8-4568-9db7-803682c4c7bd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:38.624793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d7ad8-9832-4ea8-aa71-cad176062245 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:38.654528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec23b80-9fea-467c-8c4b-98c02fb871fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:38.654528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec23b80-9fea-467c-8c4b-98c02fb871fd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.654528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec23b80-9fea-467c-8c4b-98c02fb871fd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.655015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec23b80-9fea-467c-8c4b-98c02fb871fd None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.655535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec23b80-9fea-467c-8c4b-98c02fb871fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:38.655843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec23b80-9fea-467c-8c4b-98c02fb871fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:38.657913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec23b80-9fea-467c-8c4b-98c02fb871fd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:38.668421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34004211-7f29-4299-8a73-af03f1886972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:38.669077 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34004211-7f29-4299-8a73-af03f1886972 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.669819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34004211-7f29-4299-8a73-af03f1886972 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.669819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34004211-7f29-4299-8a73-af03f1886972 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.670506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34004211-7f29-4299-8a73-af03f1886972 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=5368284e42d84e77a6fdb88c2e8dcd3d) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:38.670822 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34004211-7f29-4299-8a73-af03f1886972 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:38.673008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34004211-7f29-4299-8a73-af03f1886972 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:38.675383 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 353/710] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:38.678171 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc38dc5-2f27-48ca-8db2-6841c7e41a81 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.678171 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc38dc5-2f27-48ca-8db2-6841c7e41a81 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.678171 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc38dc5-2f27-48ca-8db2-6841c7e41a81 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.686088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 358/712] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:38 2022] GET /identity/v3/role_assignments?user.id=5368284e42d84e77a6fdb88c2e8dcd3d&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:38.686088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34004211-7f29-4299-8a73-af03f1886972 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:38.686420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 354/712] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:38.686420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc38dc5-2f27-48ca-8db2-6841c7e41a81 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:38.697964 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:38.837642 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34004211-7f29-4299-8a73-af03f1886972 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:38.838195 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc38dc5-2f27-48ca-8db2-6841c7e41a81 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:38.865857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:38.881015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f19aa0-23df-4186-af77-38970d0df84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:38.881746 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f19aa0-23df-4186-af77-38970d0df84b None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.882287 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f19aa0-23df-4186-af77-38970d0df84b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.882652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f19aa0-23df-4186-af77-38970d0df84b None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.883371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f19aa0-23df-4186-af77-38970d0df84b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:38.884899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7ba70-51e2-4b4b-aa33-54177e5abfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:38.885039 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f19aa0-23df-4186-af77-38970d0df84b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:38.885928 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7ba70-51e2-4b4b-aa33-54177e5abfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.886491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7ba70-51e2-4b4b-aa33-54177e5abfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.886994 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7ba70-51e2-4b4b-aa33-54177e5abfc6 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.887939 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7ba70-51e2-4b4b-aa33-54177e5abfc6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:38.888511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7ba70-51e2-4b4b-aa33-54177e5abfc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:38.890181 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7ba70-51e2-4b4b-aa33-54177e5abfc6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:38.896999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 359/713] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:51:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:38.898775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400a1e9-8542-4f67-bc76-2a84b0e270a2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:38.899310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400a1e9-8542-4f67-bc76-2a84b0e270a2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:38.899798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400a1e9-8542-4f67-bc76-2a84b0e270a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:38.918357 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0400a1e9-8542-4f67-bc76-2a84b0e270a2 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:38.919565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0400a1e9-8542-4f67-bc76-2a84b0e270a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:38.992212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 360/714] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:38.992212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400a1e9-8542-4f67-bc76-2a84b0e270a2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:39.002873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:39.022175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 355/715] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:51:38 2022] POST /identity/v3/users => generated 254 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:39.029195 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f19aa0-23df-4186-af77-38970d0df84b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:39.206684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400a1e9-8542-4f67-bc76-2a84b0e270a2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:39.206866 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f19aa0-23df-4186-af77-38970d0df84b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:39.256838 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d1458-1f44-428e-984e-1a85ecfb7386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:39.257991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d1458-1f44-428e-984e-1a85ecfb7386 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:39.258307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d1458-1f44-428e-984e-1a85ecfb7386 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:39.258985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d1458-1f44-428e-984e-1a85ecfb7386 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:39.259660 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d1458-1f44-428e-984e-1a85ecfb7386 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:39.261909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d1458-1f44-428e-984e-1a85ecfb7386 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:39.267067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a090c-c51c-4c09-bbd7-23ab8c54c815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:39.269340 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a090c-c51c-4c09-bbd7-23ab8c54c815 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:39.269830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a090c-c51c-4c09-bbd7-23ab8c54c815 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:39.270321 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a090c-c51c-4c09-bbd7-23ab8c54c815 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:39.271226 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a090c-c51c-4c09-bbd7-23ab8c54c815 None admin] RBAC: Adding query filter params (type=network) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:39.271746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a090c-c51c-4c09-bbd7-23ab8c54c815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:39.273483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a090c-c51c-4c09-bbd7-23ab8c54c815 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-6c0d1458-1f44-428e-984e-1a85ecfb7386 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:39.279842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:39.280870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:51:39.281706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:39.286721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 356/717] 10.222.0.88 () {56 vars in 1129 bytes} [Thu Jun 16 14:51:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:39.286974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 361/717] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:51:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:39.292364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d1458-1f44-428e-984e-1a85ecfb7386 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:39.350021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:39.417393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d1458-1f44-428e-984e-1a85ecfb7386 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:39.459608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ce4f7-0c7f-4af2-9717-c3ffcb37e4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:39.460734 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ce4f7-0c7f-4af2-9717-c3ffcb37e4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:39.461247 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ce4f7-0c7f-4af2-9717-c3ffcb37e4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:39.461725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ce4f7-0c7f-4af2-9717-c3ffcb37e4f1 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:39.462652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ce4f7-0c7f-4af2-9717-c3ffcb37e4f1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:39.463611 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ce4f7-0c7f-4af2-9717-c3ffcb37e4f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:39.465232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ce4f7-0c7f-4af2-9717-c3ffcb37e4f1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:39.471786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 357/718] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 14:51:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:39.480455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a090c-c51c-4c09-bbd7-23ab8c54c815 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:39.572329 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:39.608281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a090c-c51c-4c09-bbd7-23ab8c54c815 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:39.647768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcedfab-07a8-4efd-b284-bb8f29c8d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:39.648469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcedfab-07a8-4efd-b284-bb8f29c8d670 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:39.648760 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcedfab-07a8-4efd-b284-bb8f29c8d670 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:39.649093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcedfab-07a8-4efd-b284-bb8f29c8d670 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:39.649915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcedfab-07a8-4efd-b284-bb8f29c8d670 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:39.650281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcedfab-07a8-4efd-b284-bb8f29c8d670 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:39.651671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcedfab-07a8-4efd-b284-bb8f29c8d670 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:39.656708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 362/719] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 14:51:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:41.140617 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352adaa-749e-4a2c-8170-7c2f8e850588 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:41.141149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352adaa-749e-4a2c-8170-7c2f8e850588 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:41.141262 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352adaa-749e-4a2c-8170-7c2f8e850588 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:41.143637 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 358/720] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:41.149447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28618c2-e437-459e-89fd-af185ca15709 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:41.150283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28618c2-e437-459e-89fd-af185ca15709 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:41.150817 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28618c2-e437-459e-89fd-af185ca15709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:41.174315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e28618c2-e437-459e-89fd-af185ca15709 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:41.175857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e28618c2-e437-459e-89fd-af185ca15709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:41.207632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f47b57-5ac7-445f-847b-1e328c10a892 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:41.208312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f47b57-5ac7-445f-847b-1e328c10a892 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:41.208797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f47b57-5ac7-445f-847b-1e328c10a892 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:41.210593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 359/721] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:41.219096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20a90c-77f3-4ab4-8d55-9914a24250d1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:41.219620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20a90c-77f3-4ab4-8d55-9914a24250d1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:41.220100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20a90c-77f3-4ab4-8d55-9914a24250d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:41.238386 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5d20a90c-77f3-4ab4-8d55-9914a24250d1 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:41.239903 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5d20a90c-77f3-4ab4-8d55-9914a24250d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:41.273910 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 363/722] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:41.297177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28618c2-e437-459e-89fd-af185ca15709 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:41.380643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:41.399496 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 360/723] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:51:41.412022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20a90c-77f3-4ab4-8d55-9914a24250d1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:41.472578 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:41.479819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28618c2-e437-459e-89fd-af185ca15709 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:41.517961 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebdca-de7e-4842-b4c7-577558941fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:41.519013 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebdca-de7e-4842-b4c7-577558941fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:41.519462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebdca-de7e-4842-b4c7-577558941fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:41.520075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebdca-de7e-4842-b4c7-577558941fd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:41.532011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ebdca-de7e-4842-b4c7-577558941fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fb4ebdca-de7e-4842-b4c7-577558941fd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:41.541457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:41.542634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:41.543738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:41.546484 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 364/724] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:41.554037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebdca-de7e-4842-b4c7-577558941fd1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:41.674615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20a90c-77f3-4ab4-8d55-9914a24250d1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:41.679915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebdca-de7e-4842-b4c7-577558941fd1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:41.704047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:41.722538 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7a64f-ef33-40f8-ad6d-af6e9dcecce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:41.723801 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7a64f-ef33-40f8-ad6d-af6e9dcecce6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:41.724257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7a64f-ef33-40f8-ad6d-af6e9dcecce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:41.724683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7a64f-ef33-40f8-ad6d-af6e9dcecce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:41.725580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7a64f-ef33-40f8-ad6d-af6e9dcecce6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:41.726086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7a64f-ef33-40f8-ad6d-af6e9dcecce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:41.727695 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7a64f-ef33-40f8-ad6d-af6e9dcecce6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:41.739788 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 365/725] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:41.744948 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7a64f-ef33-40f8-ad6d-af6e9dcecce6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:41.786306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d945e-c5f2-4130-b500-d96034fbf5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:41.786741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d945e-c5f2-4130-b500-d96034fbf5cb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:41.787123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d945e-c5f2-4130-b500-d96034fbf5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:41.787532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d945e-c5f2-4130-b500-d96034fbf5cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:41.819405 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d945e-c5f2-4130-b500-d96034fbf5cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-702d945e-c5f2-4130-b500-d96034fbf5cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:41.827431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:41.828770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:41.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:41.842260 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 361/726] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:41.857681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d945e-c5f2-4130-b500-d96034fbf5cb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:41.870546 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7a64f-ef33-40f8-ad6d-af6e9dcecce6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:41.912825 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef145db-4ea4-4275-8a3f-50196fccffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:41.913922 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef145db-4ea4-4275-8a3f-50196fccffec None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:41.914559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef145db-4ea4-4275-8a3f-50196fccffec None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:41.915010 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef145db-4ea4-4275-8a3f-50196fccffec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:41.928366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef145db-4ea4-4275-8a3f-50196fccffec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:41.930364 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef145db-4ea4-4275-8a3f-50196fccffec None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-eef145db-4ea4-4275-8a3f-50196fccffec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:41.953505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:41.954475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:41.956131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:41.956131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:41.956131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:41.956131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:41.959244 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 366/727] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:41.967151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef145db-4ea4-4275-8a3f-50196fccffec None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:42.031914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:42.054940 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:42.062482 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d945e-c5f2-4130-b500-d96034fbf5cb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:42.115075 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160090bb-04c1-414f-a2d0-a0e3d1847e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:42.115858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160090bb-04c1-414f-a2d0-a0e3d1847e80 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.116220 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160090bb-04c1-414f-a2d0-a0e3d1847e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.116440 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef145db-4ea4-4275-8a3f-50196fccffec None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:42.116724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160090bb-04c1-414f-a2d0-a0e3d1847e80 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.117602 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160090bb-04c1-414f-a2d0-a0e3d1847e80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:42.117981 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160090bb-04c1-414f-a2d0-a0e3d1847e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:42.120770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160090bb-04c1-414f-a2d0-a0e3d1847e80 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:42.132492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 362/728] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:42.134733 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696d66-fb1c-4312-a5cb-13ef849fcf7f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.135172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696d66-fb1c-4312-a5cb-13ef849fcf7f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.135530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696d66-fb1c-4312-a5cb-13ef849fcf7f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.137698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 363/729] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:42.142105 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa696d66-fb1c-4312-a5cb-13ef849fcf7f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:42.161934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce466c4-3bc2-4baf-a568-cdefe81b0816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:42.163470 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce466c4-3bc2-4baf-a568-cdefe81b0816 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.163958 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce466c4-3bc2-4baf-a568-cdefe81b0816 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.164469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce466c4-3bc2-4baf-a568-cdefe81b0816 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.165375 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce466c4-3bc2-4baf-a568-cdefe81b0816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:42.165931 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce466c4-3bc2-4baf-a568-cdefe81b0816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:42.167803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce466c4-3bc2-4baf-a568-cdefe81b0816 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:42.182663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 367/730] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:42.184276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096885e-94db-41d3-846f-b27c3765d983 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.184753 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096885e-94db-41d3-846f-b27c3765d983 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.185212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096885e-94db-41d3-846f-b27c3765d983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.203686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1096885e-94db-41d3-846f-b27c3765d983 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:42.205755 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1096885e-94db-41d3-846f-b27c3765d983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:42.258321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa696d66-fb1c-4312-a5cb-13ef849fcf7f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:42.288303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 368/731] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:42.288303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b4cd3-0687-4441-a9dd-e999120d2acf None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.288303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b4cd3-0687-4441-a9dd-e999120d2acf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.288303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b4cd3-0687-4441-a9dd-e999120d2acf None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.293401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 369/732] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:42.294432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b4cd3-0687-4441-a9dd-e999120d2acf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:42.298839 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e874e-9a9b-4269-959c-4815cd44e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:42.299718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e874e-9a9b-4269-959c-4815cd44e646 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.300021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e874e-9a9b-4269-959c-4815cd44e646 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.300360 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e874e-9a9b-4269-959c-4815cd44e646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.312946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e874e-9a9b-4269-959c-4815cd44e646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:42.315537 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e874e-9a9b-4269-959c-4815cd44e646 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:42.326029 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b79e874e-9a9b-4269-959c-4815cd44e646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:42.336790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:42.343337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:42.344307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:42.344307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:42.344307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:42.344307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:42.344307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 364/733] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:42.344307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f7044-7242-40d8-9cf6-30afec36fade None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.344307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f7044-7242-40d8-9cf6-30afec36fade None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.344307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f7044-7242-40d8-9cf6-30afec36fade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.360017 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-535f7044-7242-40d8-9cf6-30afec36fade None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:42.361389 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-535f7044-7242-40d8-9cf6-30afec36fade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:42.364478 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:42.410517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b4cd3-0687-4441-a9dd-e999120d2acf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:42.430138 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 365/734] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:42.431035 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f7044-7242-40d8-9cf6-30afec36fade None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:42.450655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eef75e-23c6-4f5a-ac1e-76317339ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:42.451373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eef75e-23c6-4f5a-ac1e-76317339ab98 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.451664 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eef75e-23c6-4f5a-ac1e-76317339ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.451947 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eef75e-23c6-4f5a-ac1e-76317339ab98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.460165 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eef75e-23c6-4f5a-ac1e-76317339ab98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:42.461697 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eef75e-23c6-4f5a-ac1e-76317339ab98 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f4eef75e-23c6-4f5a-ac1e-76317339ab98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:42.476116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:42.477177 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 370/735] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 14:51:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:42.478143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eef75e-23c6-4f5a-ac1e-76317339ab98 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:42.543392 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f7044-7242-40d8-9cf6-30afec36fade None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:42.570107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:42.578061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8c80-a34c-48ec-9a18-96c24fa8a779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:42.578726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8c80-a34c-48ec-9a18-96c24fa8a779 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.579018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8c80-a34c-48ec-9a18-96c24fa8a779 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.579303 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8c80-a34c-48ec-9a18-96c24fa8a779 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.582295 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8c80-a34c-48ec-9a18-96c24fa8a779 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:42.582295 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8c80-a34c-48ec-9a18-96c24fa8a779 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:42.585962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eef75e-23c6-4f5a-ac1e-76317339ab98 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:42.620347 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36cf05-a1a9-4967-bf57-f077413bbffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:42.621003 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36cf05-a1a9-4967-bf57-f077413bbffe None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.621291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36cf05-a1a9-4967-bf57-f077413bbffe None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.621571 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36cf05-a1a9-4967-bf57-f077413bbffe None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.622287 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36cf05-a1a9-4967-bf57-f077413bbffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:42.622691 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36cf05-a1a9-4967-bf57-f077413bbffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:42.624050 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36cf05-a1a9-4967-bf57-f077413bbffe None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:42.636090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 371/736] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:42.637347 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3a4cf-0ec1-4929-a3d0-3a02e9cd67e8 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.637636 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3a4cf-0ec1-4929-a3d0-3a02e9cd67e8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.637916 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3a4cf-0ec1-4929-a3d0-3a02e9cd67e8 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.639157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 372/737] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:42.639890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3a4cf-0ec1-4929-a3d0-3a02e9cd67e8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:42.740588 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3a4cf-0ec1-4929-a3d0-3a02e9cd67e8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:42.773779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37f563-ee73-426a-8ce0-8603caf140ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:42.774566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37f563-ee73-426a-8ce0-8603caf140ed None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.774854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37f563-ee73-426a-8ce0-8603caf140ed None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.775132 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37f563-ee73-426a-8ce0-8603caf140ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.780469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:42.785453 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37f563-ee73-426a-8ce0-8603caf140ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:42.791689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 366/738] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:51:42 2022] POST /identity/v3/services => generated 240 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:42.791689 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8c80-a34c-48ec-9a18-96c24fa8a779 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ae37f563-ee73-426a-8ce0-8603caf140ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:42.793791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:42.794798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:42.795717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:42.801041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 373/739] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:42.802043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37f563-ee73-426a-8ce0-8603caf140ed None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:42.807454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:42.914847 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37f563-ee73-426a-8ce0-8603caf140ed None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:42.919085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8c80-a34c-48ec-9a18-96c24fa8a779 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:42.954954 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e51fb3-d9ce-4ae7-b19f-522828649499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:42.956747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e51fb3-d9ce-4ae7-b19f-522828649499 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.956747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e51fb3-d9ce-4ae7-b19f-522828649499 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.956941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e51fb3-d9ce-4ae7-b19f-522828649499 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.964273 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a90526-ff79-454f-bc9a-f404e1f01c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:42.965034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a90526-ff79-454f-bc9a-f404e1f01c8f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.966137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a90526-ff79-454f-bc9a-f404e1f01c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.966137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a90526-ff79-454f-bc9a-f404e1f01c8f None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.966453 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a90526-ff79-454f-bc9a-f404e1f01c8f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:42.967407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a90526-ff79-454f-bc9a-f404e1f01c8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:42.967514 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e51fb3-d9ce-4ae7-b19f-522828649499 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:42.968214 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a90526-ff79-454f-bc9a-f404e1f01c8f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:42.968341 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e51fb3-d9ce-4ae7-b19f-522828649499 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-29e51fb3-d9ce-4ae7-b19f-522828649499 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:42.986738 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:42.987759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:42.988768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 374/740] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:42.989362 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb2074-de13-41f9-a6df-5bd0e9e4bb23 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:42.989362 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb2074-de13-41f9-a6df-5bd0e9e4bb23 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:42.990017 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb2074-de13-41f9-a6df-5bd0e9e4bb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:42.998965 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 367/741] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 14:51:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:42.998965 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a90526-ff79-454f-bc9a-f404e1f01c8f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:43.007464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-dfcb2074-de13-41f9-a6df-5bd0e9e4bb23 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:43.008680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-dfcb2074-de13-41f9-a6df-5bd0e9e4bb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:43.057084 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:43.081048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:43.096382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 375/742] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:42 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:51:43.096382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb2074-de13-41f9-a6df-5bd0e9e4bb23 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:43.110719 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a90526-ff79-454f-bc9a-f404e1f01c8f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:43.148832 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d6fbc-418f-4215-adbf-3e172145221a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:43.149651 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d6fbc-418f-4215-adbf-3e172145221a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:43.149945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d6fbc-418f-4215-adbf-3e172145221a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:43.150228 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d6fbc-418f-4215-adbf-3e172145221a None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:43.151031 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d6fbc-418f-4215-adbf-3e172145221a None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:43.151347 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d6fbc-418f-4215-adbf-3e172145221a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:43.152769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d6fbc-418f-4215-adbf-3e172145221a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:43.167026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 368/743] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:43.167026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d6fbc-418f-4215-adbf-3e172145221a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:43.218052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb2074-de13-41f9-a6df-5bd0e9e4bb23 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:43.257219 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bccb1-eb4f-4d2e-8ca0-9c49b2577d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:43.257828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bccb1-eb4f-4d2e-8ca0-9c49b2577d1f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:43.258103 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bccb1-eb4f-4d2e-8ca0-9c49b2577d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:43.258383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bccb1-eb4f-4d2e-8ca0-9c49b2577d1f None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:43.259217 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bccb1-eb4f-4d2e-8ca0-9c49b2577d1f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:43.259524 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bccb1-eb4f-4d2e-8ca0-9c49b2577d1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:43.260980 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bccb1-eb4f-4d2e-8ca0-9c49b2577d1f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:43.278201 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d6fbc-418f-4215-adbf-3e172145221a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:43.281322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 376/744] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 14:51:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:43.282298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bccb1-eb4f-4d2e-8ca0-9c49b2577d1f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:43.301587 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:43.319772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c4119-a80a-4bb1-988e-65dd1e1d7c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:43.320738 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c4119-a80a-4bb1-988e-65dd1e1d7c8a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:43.320888 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c4119-a80a-4bb1-988e-65dd1e1d7c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:43.322901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c4119-a80a-4bb1-988e-65dd1e1d7c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:43.336186 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c4119-a80a-4bb1-988e-65dd1e1d7c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:43.337652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c4119-a80a-4bb1-988e-65dd1e1d7c8a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:43.341323 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-e39c4119-a80a-4bb1-988e-65dd1e1d7c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:43.355990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:43.356971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:43.357870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:43.357870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:43.357870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:43.357870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:43.357870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 369/745] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:43.358147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c4119-a80a-4bb1-988e-65dd1e1d7c8a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:43.395465 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bccb1-eb4f-4d2e-8ca0-9c49b2577d1f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:43.433821 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac560d-d853-416a-99f5-41dd0d418de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:43.434494 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac560d-d853-416a-99f5-41dd0d418de8 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:43.434786 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac560d-d853-416a-99f5-41dd0d418de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:43.435074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac560d-d853-416a-99f5-41dd0d418de8 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:43.435799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac560d-d853-416a-99f5-41dd0d418de8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:43.437210 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac560d-d853-416a-99f5-41dd0d418de8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:43.464882 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c4119-a80a-4bb1-988e-65dd1e1d7c8a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:43.502194 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921de5f3-030f-47d0-a245-8343a7247cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:43.503127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921de5f3-030f-47d0-a245-8343a7247cdd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:43.503414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921de5f3-030f-47d0-a245-8343a7247cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:43.503715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921de5f3-030f-47d0-a245-8343a7247cdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:43.515263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921de5f3-030f-47d0-a245-8343a7247cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:43.516771 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921de5f3-030f-47d0-a245-8343a7247cdd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-921de5f3-030f-47d0-a245-8343a7247cdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:43.536250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:43.537371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:43.538304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:43.538304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:43.538304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:43.538304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:43.538304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 370/746] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:43.538304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921de5f3-030f-47d0-a245-8343a7247cdd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:43.556281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 377/747] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:51:43 2022] POST /identity/v3/services => generated 251 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:43.556281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac560d-d853-416a-99f5-41dd0d418de8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:43.564473 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:43.697096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921de5f3-030f-47d0-a245-8343a7247cdd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:43.728154 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac560d-d853-416a-99f5-41dd0d418de8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:43.741863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:43.743813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbdfa3-eaa0-4843-9515-bccda042b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:43.744616 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbdfa3-eaa0-4843-9515-bccda042b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:43.745003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbdfa3-eaa0-4843-9515-bccda042b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:43.745398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbdfa3-eaa0-4843-9515-bccda042b4d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:43.771889 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7aa3-e557-4464-b8bb-26f6c107ba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:43.772970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7aa3-e557-4464-b8bb-26f6c107ba72 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:43.773281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7aa3-e557-4464-b8bb-26f6c107ba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:43.773740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7aa3-e557-4464-b8bb-26f6c107ba72 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:43.774683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d7aa3-e557-4464-b8bb-26f6c107ba72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:43.775168 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d7aa3-e557-4464-b8bb-26f6c107ba72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:43.777164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d7aa3-e557-4464-b8bb-26f6c107ba72 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:43.794155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 378/748] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:43.795146 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7aa3-e557-4464-b8bb-26f6c107ba72 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:43.834315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbdfa3-eaa0-4843-9515-bccda042b4d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:43.835915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbdfa3-eaa0-4843-9515-bccda042b4d2 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:43.868694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:43.933244 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7aa3-e557-4464-b8bb-26f6c107ba72 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:43.969944 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 371/749] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:51:43.969944 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbdfa3-eaa0-4843-9515-bccda042b4d2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:43.990124 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2817ef-5f0f-40d7-a0e3-2ab70d512f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:43.990840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2817ef-5f0f-40d7-a0e3-2ab70d512f49 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:43.991134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2817ef-5f0f-40d7-a0e3-2ab70d512f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:43.991418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2817ef-5f0f-40d7-a0e3-2ab70d512f49 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:43.992150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2817ef-5f0f-40d7-a0e3-2ab70d512f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:43.992502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2817ef-5f0f-40d7-a0e3-2ab70d512f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:43.993914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2817ef-5f0f-40d7-a0e3-2ab70d512f49 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:44.012689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 379/750] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:44.013569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2817ef-5f0f-40d7-a0e3-2ab70d512f49 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:44.045714 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:44.099449 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbdfa3-eaa0-4843-9515-bccda042b4d2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:44.121646 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:44.126058 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2817ef-5f0f-40d7-a0e3-2ab70d512f49 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:44.137308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34621abf-05f9-48a3-8350-b2018a6c17f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:44.138115 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621abf-05f9-48a3-8350-b2018a6c17f4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:44.138400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621abf-05f9-48a3-8350-b2018a6c17f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:44.138675 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621abf-05f9-48a3-8350-b2018a6c17f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:44.147495 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34621abf-05f9-48a3-8350-b2018a6c17f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:44.148977 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34621abf-05f9-48a3-8350-b2018a6c17f4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:44.165942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a735d2-378e-4d14-b75a-108583f904ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:44.167054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a735d2-378e-4d14-b75a-108583f904ac None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-34621abf-05f9-48a3-8350-b2018a6c17f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:44.167255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:44.168134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:44.169083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:44.169083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:44.169083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:44.169083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:44.169083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:44.169083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:44.169083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:44.169083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:44.170320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a735d2-378e-4d14-b75a-108583f904ac None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:44.170797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a735d2-378e-4d14-b75a-108583f904ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:44.173649 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 372/751] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:44.175469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34621abf-05f9-48a3-8350-b2018a6c17f4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:44.188175 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a735d2-378e-4d14-b75a-108583f904ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:44.189866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a735d2-378e-4d14-b75a-108583f904ac None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-62a735d2-378e-4d14-b75a-108583f904ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:44.213475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:44.214451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:44.215359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:44.215359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:44.215359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:44.215359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:44.215359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 380/752] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:44.216162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a735d2-378e-4d14-b75a-108583f904ac None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:44.296625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34621abf-05f9-48a3-8350-b2018a6c17f4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:44.320888 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:44.338176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672548d6-d181-4e51-a245-1eb51b79ccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:44.339031 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672548d6-d181-4e51-a245-1eb51b79ccdd None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:44.339357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672548d6-d181-4e51-a245-1eb51b79ccdd None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:44.339610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672548d6-d181-4e51-a245-1eb51b79ccdd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:44.348584 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672548d6-d181-4e51-a245-1eb51b79ccdd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:44.350048 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672548d6-d181-4e51-a245-1eb51b79ccdd None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:44.350992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a735d2-378e-4d14-b75a-108583f904ac None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-672548d6-d181-4e51-a245-1eb51b79ccdd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:44.367385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:44.368445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:51:44.369440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:44.373234 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 373/753] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:51:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:44.374188 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672548d6-d181-4e51-a245-1eb51b79ccdd None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:44.401034 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5cb1f-3308-4859-9ab7-2da9687e638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:44.402441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5cb1f-3308-4859-9ab7-2da9687e638a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:44.402932 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5cb1f-3308-4859-9ab7-2da9687e638a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:44.403436 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5cb1f-3308-4859-9ab7-2da9687e638a None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:44.405107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5cb1f-3308-4859-9ab7-2da9687e638a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:44.405645 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5cb1f-3308-4859-9ab7-2da9687e638a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:44.407923 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5cb1f-3308-4859-9ab7-2da9687e638a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:44.432360 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 381/754] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:44.432360 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5cb1f-3308-4859-9ab7-2da9687e638a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:44.434313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:44.504940 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672548d6-d181-4e51-a245-1eb51b79ccdd None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:44.545692 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cea44-de0b-44e0-bb18-d3ea4ff5e6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:44.546377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cea44-de0b-44e0-bb18-d3ea4ff5e6ba None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:44.546723 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cea44-de0b-44e0-bb18-d3ea4ff5e6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:44.547052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cea44-de0b-44e0-bb18-d3ea4ff5e6ba None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:44.547792 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cea44-de0b-44e0-bb18-d3ea4ff5e6ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:44.548108 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cea44-de0b-44e0-bb18-d3ea4ff5e6ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:44.549623 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cea44-de0b-44e0-bb18-d3ea4ff5e6ba None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:44.559867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5cb1f-3308-4859-9ab7-2da9687e638a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:44.567872 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 374/755] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:44.568828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cea44-de0b-44e0-bb18-d3ea4ff5e6ba None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:44.600278 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bde15-fa6a-4ece-8c95-bb24618b5a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:44.601020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bde15-fa6a-4ece-8c95-bb24618b5a6c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:44.601477 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bde15-fa6a-4ece-8c95-bb24618b5a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:44.601766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bde15-fa6a-4ece-8c95-bb24618b5a6c None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:44.603124 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bde15-fa6a-4ece-8c95-bb24618b5a6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:44.603562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bde15-fa6a-4ece-8c95-bb24618b5a6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:44.605598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bde15-fa6a-4ece-8c95-bb24618b5a6c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:44.617796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:44.632309 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 382/756] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 14:51:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:44.633469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bde15-fa6a-4ece-8c95-bb24618b5a6c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:44.704119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:44.708069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cea44-de0b-44e0-bb18-d3ea4ff5e6ba None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:44.758398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca14e4b-9239-46f0-a7a2-961364d913e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:44.759726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca14e4b-9239-46f0-a7a2-961364d913e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:44.760024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca14e4b-9239-46f0-a7a2-961364d913e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:44.760340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca14e4b-9239-46f0-a7a2-961364d913e7 None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/5368284e42d84e77a6fdb88c2e8dcd3d/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:44.763323 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca14e4b-9239-46f0-a7a2-961364d913e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=5368284e42d84e77a6fdb88c2e8dcd3d, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:44.765127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca14e4b-9239-46f0-a7a2-961364d913e7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:44.770785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bde15-fa6a-4ece-8c95-bb24618b5a6c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:44.810095 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99d089-44ff-4676-a646-0a5c1528737d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:44.810860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d089-44ff-4676-a646-0a5c1528737d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:44.811165 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d089-44ff-4676-a646-0a5c1528737d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:44.811432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d089-44ff-4676-a646-0a5c1528737d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:44.818764 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99d089-44ff-4676-a646-0a5c1528737d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:44.820265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99d089-44ff-4676-a646-0a5c1528737d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-5c99d089-44ff-4676-a646-0a5c1528737d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:44.836037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:44.837291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:44.838216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:44.838216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:44.838216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:44.838216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:44.838216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:44.838216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:44.838216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:44.838216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:44.839289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 383/757] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:44.840464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99d089-44ff-4676-a646-0a5c1528737d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:44.865992 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 375/758] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:51:44 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/5368284e42d84e77a6fdb88c2e8dcd3d/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:44.867159 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca14e4b-9239-46f0-a7a2-961364d913e7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:44.942577 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:45.024176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca14e4b-9239-46f0-a7a2-961364d913e7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:45.050196 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99d089-44ff-4676-a646-0a5c1528737d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:45.078370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b855773-33e0-4350-bd10-b17f1ffa7bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:45.079393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b855773-33e0-4350-bd10-b17f1ffa7bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:45.079814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b855773-33e0-4350-bd10-b17f1ffa7bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:45.080283 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b855773-33e0-4350-bd10-b17f1ffa7bf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:45.081046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b855773-33e0-4350-bd10-b17f1ffa7bf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:45.082006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b855773-33e0-4350-bd10-b17f1ffa7bf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:45.083975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b855773-33e0-4350-bd10-b17f1ffa7bf8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:45.104180 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 376/759] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:45.106284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b855773-33e0-4350-bd10-b17f1ffa7bf8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:45.115498 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59262840-39f8-4039-8108-fba2374c388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:45.116212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59262840-39f8-4039-8108-fba2374c388f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:45.116521 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59262840-39f8-4039-8108-fba2374c388f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:45.116800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59262840-39f8-4039-8108-fba2374c388f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:45.139645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:45.207176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59262840-39f8-4039-8108-fba2374c388f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:45.209180 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59262840-39f8-4039-8108-fba2374c388f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:45.240056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b855773-33e0-4350-bd10-b17f1ffa7bf8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:45.278239 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:45.286448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26f5a0-25d7-489b-8a20-a83a0bf43b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:45.287126 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26f5a0-25d7-489b-8a20-a83a0bf43b3f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:45.287432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26f5a0-25d7-489b-8a20-a83a0bf43b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:45.288053 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26f5a0-25d7-489b-8a20-a83a0bf43b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:45.288574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26f5a0-25d7-489b-8a20-a83a0bf43b3f None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=bf0dd8c9980e4640aafd34946c4193a3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:45.288889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26f5a0-25d7-489b-8a20-a83a0bf43b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:45.290423 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26f5a0-25d7-489b-8a20-a83a0bf43b3f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:45.297442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 377/760] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:45 2022] GET /identity/v3/role_assignments?user.id=bf0dd8c9980e4640aafd34946c4193a3&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:45.298794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d488d2-4b8b-4261-970d-ef25c3ed926d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:45.299088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d488d2-4b8b-4261-970d-ef25c3ed926d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:45.299444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d488d2-4b8b-4261-970d-ef25c3ed926d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:45.313364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 378/761] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:45 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:45.319196 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab2377-cbe6-49b0-87a7-78c50a30ef42 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:45.319532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab2377-cbe6-49b0-87a7-78c50a30ef42 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:45.319759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab2377-cbe6-49b0-87a7-78c50a30ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:45.361931 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-40ab2377-cbe6-49b0-87a7-78c50a30ef42 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:45.363072 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-40ab2377-cbe6-49b0-87a7-78c50a30ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:45.365508 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 384/762] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:51:45.382751 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59262840-39f8-4039-8108-fba2374c388f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:45.523261 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59262840-39f8-4039-8108-fba2374c388f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:45.552354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 379/763] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:51:45.592951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a9b99-1971-44c7-8bbb-94bb27185d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:45.594165 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a9b99-1971-44c7-8bbb-94bb27185d85 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:45.594681 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a9b99-1971-44c7-8bbb-94bb27185d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:45.595214 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a9b99-1971-44c7-8bbb-94bb27185d85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:45.615078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a9b99-1971-44c7-8bbb-94bb27185d85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:45.617078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a9b99-1971-44c7-8bbb-94bb27185d85 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:45.619091 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab2377-cbe6-49b0-87a7-78c50a30ef42 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:45.652533 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a80a9b99-1971-44c7-8bbb-94bb27185d85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:45.666758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:45.667777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:45.668737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:45.668737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:45.668737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:45.668737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:45.675678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 385/764] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:45.677275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a9b99-1971-44c7-8bbb-94bb27185d85 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:45.691857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:45.791961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab2377-cbe6-49b0-87a7-78c50a30ef42 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:45.836112 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a9b99-1971-44c7-8bbb-94bb27185d85 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:45.882511 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7cdd4-b1f9-4eba-bf0d-9642d2bd115e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:45.883268 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7cdd4-b1f9-4eba-bf0d-9642d2bd115e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:45.883554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7cdd4-b1f9-4eba-bf0d-9642d2bd115e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:45.883942 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7cdd4-b1f9-4eba-bf0d-9642d2bd115e None admin] PATH_INFO: `/v3/services/network` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:45.884697 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7cdd4-b1f9-4eba-bf0d-9642d2bd115e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:45.886342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7cdd4-b1f9-4eba-bf0d-9642d2bd115e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:45.895399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4661-870b-424d-8deb-7fe6878fd92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:45.897265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4661-870b-424d-8deb-7fe6878fd92e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:45.897768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4661-870b-424d-8deb-7fe6878fd92e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:45.898252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f4661-870b-424d-8deb-7fe6878fd92e None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:45.899372 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f4661-870b-424d-8deb-7fe6878fd92e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:45.899955 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f4661-870b-424d-8deb-7fe6878fd92e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:45.903108 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f4661-870b-424d-8deb-7fe6878fd92e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-33f7cdd4-b1f9-4eba-bf0d-9642d2bd115e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:45.918314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:45.919394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:51:45.920263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:45.924734 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 380/765] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:45 2022] GET /identity/v3/services/network => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:45.931946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7cdd4-b1f9-4eba-bf0d-9642d2bd115e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:45.936185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 386/766] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:45.937369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4661-870b-424d-8deb-7fe6878fd92e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:46.067093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:46.087666 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:46.111397 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7cdd4-b1f9-4eba-bf0d-9642d2bd115e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:46.149961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa9f2d-d7fb-40e7-9846-1087a9b92ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:46.150555 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa9f2d-d7fb-40e7-9846-1087a9b92ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.150845 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa9f2d-d7fb-40e7-9846-1087a9b92ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.151129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa9f2d-d7fb-40e7-9846-1087a9b92ad0 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.151842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa9f2d-d7fb-40e7-9846-1087a9b92ad0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:46.152152 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa9f2d-d7fb-40e7-9846-1087a9b92ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:46.153553 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa9f2d-d7fb-40e7-9846-1087a9b92ad0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:46.157573 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f4661-870b-424d-8deb-7fe6878fd92e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:46.172783 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 381/767] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:51:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:46.174334 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa9f2d-d7fb-40e7-9846-1087a9b92ad0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:46.237947 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cbde5-91ce-43e7-99ed-a0cdd7334c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:46.238762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cbde5-91ce-43e7-99ed-a0cdd7334c26 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.239045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cbde5-91ce-43e7-99ed-a0cdd7334c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.239325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cbde5-91ce-43e7-99ed-a0cdd7334c26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.257065 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cbde5-91ce-43e7-99ed-a0cdd7334c26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:46.258568 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cbde5-91ce-43e7-99ed-a0cdd7334c26 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-723cbde5-91ce-43e7-99ed-a0cdd7334c26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:46.282519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:46.284098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:46.285608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:46.285608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:46.285608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:46.285608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:46.285608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:46.285608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:46.285608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:46.285608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:46.285608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 387/768] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:46.286263 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa91e7e-c62a-4f84-938a-a7353590d832 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.287520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa91e7e-c62a-4f84-938a-a7353590d832 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.287520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa91e7e-c62a-4f84-938a-a7353590d832 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.293052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 388/769] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:46.293892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa91e7e-c62a-4f84-938a-a7353590d832 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:46.343494 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa9f2d-d7fb-40e7-9846-1087a9b92ad0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:46.390415 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627cad1-82ee-43fc-8ac7-ea49d3a7f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:46.390805 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627cad1-82ee-43fc-8ac7-ea49d3a7f803 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.391088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627cad1-82ee-43fc-8ac7-ea49d3a7f803 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.391374 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627cad1-82ee-43fc-8ac7-ea49d3a7f803 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.392699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627cad1-82ee-43fc-8ac7-ea49d3a7f803 None admin] RBAC: Adding query filter params (type=network) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:46.393016 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627cad1-82ee-43fc-8ac7-ea49d3a7f803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:46.394641 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627cad1-82ee-43fc-8ac7-ea49d3a7f803 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:46.406135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 382/770] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:51:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:46.407492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2493bfc-a6b3-46f1-b933-14b896f061a7 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.408482 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2493bfc-a6b3-46f1-b933-14b896f061a7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.408788 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2493bfc-a6b3-46f1-b933-14b896f061a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.412630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa91e7e-c62a-4f84-938a-a7353590d832 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:46.426107 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e2493bfc-a6b3-46f1-b933-14b896f061a7 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:46.427416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e2493bfc-a6b3-46f1-b933-14b896f061a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:46.427891 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:46.448687 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:46.455657 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fc9e-e405-43c9-afd2-80cb9c4d5b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:46.456499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fc9e-e405-43c9-afd2-80cb9c4d5b25 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.456669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fc9e-e405-43c9-afd2-80cb9c4d5b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.456959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fc9e-e405-43c9-afd2-80cb9c4d5b25 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.457773 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fc9e-e405-43c9-afd2-80cb9c4d5b25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:46.458096 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fc9e-e405-43c9-afd2-80cb9c4d5b25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:46.459621 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fc9e-e405-43c9-afd2-80cb9c4d5b25 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:46.477949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 389/771] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:46.478880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fc9e-e405-43c9-afd2-80cb9c4d5b25 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:46.504004 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 383/772] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:51:46.505266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2493bfc-a6b3-46f1-b933-14b896f061a7 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:46.592668 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fc9e-e405-43c9-afd2-80cb9c4d5b25 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:46.616644 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2493bfc-a6b3-46f1-b933-14b896f061a7 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:46.631097 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f173f67-8093-4cfd-9e28-a650f671c207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:46.631793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f173f67-8093-4cfd-9e28-a650f671c207 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.632362 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f173f67-8093-4cfd-9e28-a650f671c207 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.632723 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f173f67-8093-4cfd-9e28-a650f671c207 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.633469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f173f67-8093-4cfd-9e28-a650f671c207 None admin] RBAC: Adding query filter params (interface=public, service_id=bdace06fa009419c86053b994a6edca5, region_id=RegionOne) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:46.633827 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f173f67-8093-4cfd-9e28-a650f671c207 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:46.635350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f173f67-8093-4cfd-9e28-a650f671c207 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:46.649441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 390/773] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 14:51:46 2022] GET /identity/v3/endpoints?service_id=bdace06fa009419c86053b994a6edca5&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:46.650442 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f173f67-8093-4cfd-9e28-a650f671c207 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:46.660684 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c4cd4-b288-4a4e-9641-69eeb397ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:46.661703 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c4cd4-b288-4a4e-9641-69eeb397ba6d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.662210 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c4cd4-b288-4a4e-9641-69eeb397ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.662691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c4cd4-b288-4a4e-9641-69eeb397ba6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.663799 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c4cd4-b288-4a4e-9641-69eeb397ba6d None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=eadd75d096ac49758fdf012c8d8755ad) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:46.664394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c4cd4-b288-4a4e-9641-69eeb397ba6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:46.666088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c4cd4-b288-4a4e-9641-69eeb397ba6d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:46.669881 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:46.679574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 384/774] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:46 2022] GET /identity/v3/role_assignments?user.id=eadd75d096ac49758fdf012c8d8755ad&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:46.679574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c4cd4-b288-4a4e-9641-69eeb397ba6d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:46.752514 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:46.857792 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c4cd4-b288-4a4e-9641-69eeb397ba6d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:46.868462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f173f67-8093-4cfd-9e28-a650f671c207 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:46.922428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b88ea6-2d85-4d0b-9864-89afafa16713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:46.923666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b88ea6-2d85-4d0b-9864-89afafa16713 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.924210 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b88ea6-2d85-4d0b-9864-89afafa16713 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.924757 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b88ea6-2d85-4d0b-9864-89afafa16713 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.925823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b88ea6-2d85-4d0b-9864-89afafa16713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:46.927157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161c722-0c95-4863-83dc-7de2efe18196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:46.927698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b88ea6-2d85-4d0b-9864-89afafa16713 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:46.928200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161c722-0c95-4863-83dc-7de2efe18196 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:46.928614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161c722-0c95-4863-83dc-7de2efe18196 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:46.928904 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161c722-0c95-4863-83dc-7de2efe18196 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:46.929581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161c722-0c95-4863-83dc-7de2efe18196 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:46.933452 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161c722-0c95-4863-83dc-7de2efe18196 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:46.942766 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 385/775] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 14:51:46 2022] GET /identity/v3/services => generated 772 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f161c722-0c95-4863-83dc-7de2efe18196 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:46.952308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:46.953334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:51:46.954176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:46.957541 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 391/776] 10.222.0.88 () {56 vars in 1129 bytes} [Thu Jun 16 14:51:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:46.963641 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b88ea6-2d85-4d0b-9864-89afafa16713 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:47.050764 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:47.099624 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b88ea6-2d85-4d0b-9864-89afafa16713 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:47.153714 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7a6ec-2350-4318-9e0b-8c50fd98564f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:47.154407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7a6ec-2350-4318-9e0b-8c50fd98564f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:47.154695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7a6ec-2350-4318-9e0b-8c50fd98564f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:47.154978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7a6ec-2350-4318-9e0b-8c50fd98564f None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:47.155702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7a6ec-2350-4318-9e0b-8c50fd98564f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:47.156016 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7a6ec-2350-4318-9e0b-8c50fd98564f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:47.157455 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7a6ec-2350-4318-9e0b-8c50fd98564f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:47.170117 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 386/777] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 14:51:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:47.176412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161c722-0c95-4863-83dc-7de2efe18196 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:47.293763 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161c722-0c95-4863-83dc-7de2efe18196 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:47.312893 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:47.334487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cc868-8da7-4364-8043-fe1f7449e83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:47.335258 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cc868-8da7-4364-8043-fe1f7449e83c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:47.335559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cc868-8da7-4364-8043-fe1f7449e83c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:47.335855 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cc868-8da7-4364-8043-fe1f7449e83c None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:47.336657 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cc868-8da7-4364-8043-fe1f7449e83c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:47.336966 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cc868-8da7-4364-8043-fe1f7449e83c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:47.338365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cc868-8da7-4364-8043-fe1f7449e83c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:47.344255 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 392/778] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 14:51:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:47.350262 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7a6ec-2350-4318-9e0b-8c50fd98564f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:47.516359 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7a6ec-2350-4318-9e0b-8c50fd98564f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:47.559522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:47.578611 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe52c7c-db60-4cc7-90e2-1f53c8319d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:47.578611 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe52c7c-db60-4cc7-90e2-1f53c8319d06 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:47.578611 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe52c7c-db60-4cc7-90e2-1f53c8319d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:47.578611 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe52c7c-db60-4cc7-90e2-1f53c8319d06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:47.579119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe52c7c-db60-4cc7-90e2-1f53c8319d06 None admin] RBAC: Adding query filter params (interface=public, service_id=6666e14e5a2b4b6e87dfc226a1dc5d5f, region_id=RegionOne) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:47.579119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe52c7c-db60-4cc7-90e2-1f53c8319d06 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:47.579119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe52c7c-db60-4cc7-90e2-1f53c8319d06 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:47.591802 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 387/779] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 14:51:47 2022] GET /identity/v3/endpoints?service_id=6666e14e5a2b4b6e87dfc226a1dc5d5f&interface=public®ion_id=RegionOne => generated 194 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:47.598241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cc868-8da7-4364-8043-fe1f7449e83c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:47.779492 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cc868-8da7-4364-8043-fe1f7449e83c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:47.794084 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e61774-3765-44a1-92f3-e6c204292132 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:47.794379 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e61774-3765-44a1-92f3-e6c204292132 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:47.794665 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e61774-3765-44a1-92f3-e6c204292132 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:47.796197 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 388/780] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:47.804435 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1d63d-7a8b-4bac-bb35-f001a4ed9f4c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:47.804761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1d63d-7a8b-4bac-bb35-f001a4ed9f4c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:47.804996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1d63d-7a8b-4bac-bb35-f001a4ed9f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:47.837446 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-44d1d63d-7a8b-4bac-bb35-f001a4ed9f4c None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:47.838973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-44d1d63d-7a8b-4bac-bb35-f001a4ed9f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:47.859550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb8fea-2a73-4454-992d-2b2b1d7fc15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:47.860255 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8fea-2a73-4454-992d-2b2b1d7fc15d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:47.860555 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8fea-2a73-4454-992d-2b2b1d7fc15d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:47.861374 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8fea-2a73-4454-992d-2b2b1d7fc15d None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:47.862091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb8fea-2a73-4454-992d-2b2b1d7fc15d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:47.863582 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb8fea-2a73-4454-992d-2b2b1d7fc15d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:47.869411 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 393/781] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 14:51:47 2022] GET /identity/v3/services => generated 772 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:47.870783 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552819ac-5de3-4828-a43a-872256674d0f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:47.871079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552819ac-5de3-4828-a43a-872256674d0f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:47.871372 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552819ac-5de3-4828-a43a-872256674d0f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:47.872946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 394/782] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:47.879833 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1808d-5265-48b6-9c5b-bebf6e7ae5c7 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:47.880216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1808d-5265-48b6-9c5b-bebf6e7ae5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:47.880490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1808d-5265-48b6-9c5b-bebf6e7ae5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:47.904534 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-80c1808d-5265-48b6-9c5b-bebf6e7ae5c7 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:47.906588 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-80c1808d-5265-48b6-9c5b-bebf6e7ae5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:47.968310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:47.977784 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 389/783] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:51:48.012666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1d63d-7a8b-4bac-bb35-f001a4ed9f4c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:48.013957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 395/784] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:51:48.040475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1808d-5265-48b6-9c5b-bebf6e7ae5c7 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:48.194177 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1d63d-7a8b-4bac-bb35-f001a4ed9f4c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:48.227329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:48.230763 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1808d-5265-48b6-9c5b-bebf6e7ae5c7 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:48.246748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53fda4-6adc-4db9-b54f-ff842ccf9a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:48.248868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53fda4-6adc-4db9-b54f-ff842ccf9a12 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:48.248868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53fda4-6adc-4db9-b54f-ff842ccf9a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:48.248868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53fda4-6adc-4db9-b54f-ff842ccf9a12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:48.263815 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53fda4-6adc-4db9-b54f-ff842ccf9a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:48.272976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4adc6-b69a-4249-9517-aab2282c279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:48.273860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4adc6-b69a-4249-9517-aab2282c279d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:48.274213 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4adc6-b69a-4249-9517-aab2282c279d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:48.274497 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4adc6-b69a-4249-9517-aab2282c279d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-0b53fda4-6adc-4db9-b54f-ff842ccf9a12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:48.275658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:48.276618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:48.277476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:48.280493 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 390/785] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:48.288514 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53fda4-6adc-4db9-b54f-ff842ccf9a12 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:48.291411 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4adc6-b69a-4249-9517-aab2282c279d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-28d4adc6-b69a-4249-9517-aab2282c279d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:48.301242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:48.302252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:48.303135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:48.308151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 396/786] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:48.309610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4adc6-b69a-4249-9517-aab2282c279d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:48.353796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:48.446644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4adc6-b69a-4249-9517-aab2282c279d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:48.452473 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53fda4-6adc-4db9-b54f-ff842ccf9a12 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:48.495383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af93e9-b202-4e7c-8391-127f66bf8ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:48.495879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af93e9-b202-4e7c-8391-127f66bf8ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:48.496096 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af93e9-b202-4e7c-8391-127f66bf8ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:48.496404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af93e9-b202-4e7c-8391-127f66bf8ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:48.497207 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af93e9-b202-4e7c-8391-127f66bf8ac3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:48.497519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af93e9-b202-4e7c-8391-127f66bf8ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:48.498956 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af93e9-b202-4e7c-8391-127f66bf8ac3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:48.520550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 397/787] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:48.524247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af93e9-b202-4e7c-8391-127f66bf8ac3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:48.531970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e8864-533b-44ce-aebd-b1000ecf87e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:48.532731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8864-533b-44ce-aebd-b1000ecf87e4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:48.533036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8864-533b-44ce-aebd-b1000ecf87e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:48.533323 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e8864-533b-44ce-aebd-b1000ecf87e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:48.534113 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e8864-533b-44ce-aebd-b1000ecf87e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:48.534428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e8864-533b-44ce-aebd-b1000ecf87e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:48.535868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e8864-533b-44ce-aebd-b1000ecf87e4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:48.561699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 391/788] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:48.566824 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e8864-533b-44ce-aebd-b1000ecf87e4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:48.662344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:48.700613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:48.704003 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af93e9-b202-4e7c-8391-127f66bf8ac3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:48.744340 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8320f5-d735-4070-b948-7f3eaabaaa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:48.745186 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8320f5-d735-4070-b948-7f3eaabaaa7d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:48.745481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8320f5-d735-4070-b948-7f3eaabaaa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:48.745773 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8320f5-d735-4070-b948-7f3eaabaaa7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:48.757818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8320f5-d735-4070-b948-7f3eaabaaa7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:48.759344 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8320f5-d735-4070-b948-7f3eaabaaa7d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-bc8320f5-d735-4070-b948-7f3eaabaaa7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:48.778026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:48.779009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:48.780335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:48.780335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:48.780335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:48.780335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:48.782224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e8864-533b-44ce-aebd-b1000ecf87e4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:48.784371 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 398/789] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:48.786803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8320f5-d735-4070-b948-7f3eaabaaa7d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:48.857041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617cf06-ea8d-4600-a93d-c73461cf032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:48.857778 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617cf06-ea8d-4600-a93d-c73461cf032d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:48.858141 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617cf06-ea8d-4600-a93d-c73461cf032d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:48.858423 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617cf06-ea8d-4600-a93d-c73461cf032d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:48.873490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617cf06-ea8d-4600-a93d-c73461cf032d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:48.874943 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617cf06-ea8d-4600-a93d-c73461cf032d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:48.909208 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8320f5-d735-4070-b948-7f3eaabaaa7d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c617cf06-ea8d-4600-a93d-c73461cf032d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:48.911982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:48.914312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:48.916739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:48.916739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:48.916739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:48.916739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:48.925771 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 392/790] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:48.937327 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617cf06-ea8d-4600-a93d-c73461cf032d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:48.948324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ceff6-bb19-4baf-b9e3-326c443ca816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:48.949142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ceff6-bb19-4baf-b9e3-326c443ca816 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:48.949468 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ceff6-bb19-4baf-b9e3-326c443ca816 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:48.949881 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ceff6-bb19-4baf-b9e3-326c443ca816 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:48.951295 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ceff6-bb19-4baf-b9e3-326c443ca816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:48.951737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ceff6-bb19-4baf-b9e3-326c443ca816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:48.954265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ceff6-bb19-4baf-b9e3-326c443ca816 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:48.975687 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 399/791] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:48.976582 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ceff6-bb19-4baf-b9e3-326c443ca816 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:49.004734 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:49.113161 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:49.118743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ceff6-bb19-4baf-b9e3-326c443ca816 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:49.119095 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617cf06-ea8d-4600-a93d-c73461cf032d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:49.158442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212b4cd-620c-4189-adf4-1b84d7b73180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:49.159784 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212b4cd-620c-4189-adf4-1b84d7b73180 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.160568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212b4cd-620c-4189-adf4-1b84d7b73180 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.162190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212b4cd-620c-4189-adf4-1b84d7b73180 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.162885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212b4cd-620c-4189-adf4-1b84d7b73180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:49.163206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212b4cd-620c-4189-adf4-1b84d7b73180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:49.165500 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212b4cd-620c-4189-adf4-1b84d7b73180 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:49.181838 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 393/792] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:49.187853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212b4cd-620c-4189-adf4-1b84d7b73180 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:49.189704 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f165a3-849b-4654-88dc-29a9f82027cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:49.191207 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f165a3-849b-4654-88dc-29a9f82027cb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.191882 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f165a3-849b-4654-88dc-29a9f82027cb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.192570 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f165a3-849b-4654-88dc-29a9f82027cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.206587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f165a3-849b-4654-88dc-29a9f82027cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:49.209375 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f165a3-849b-4654-88dc-29a9f82027cb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-28f165a3-849b-4654-88dc-29a9f82027cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:49.243648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:49.244903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:49.246023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:49.258485 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 400/793] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:49.265179 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f165a3-849b-4654-88dc-29a9f82027cb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:49.333761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212b4cd-620c-4189-adf4-1b84d7b73180 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:49.373704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d7e90-9fe7-4261-91ec-55499a623fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:49.374623 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d7e90-9fe7-4261-91ec-55499a623fbf None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.375092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d7e90-9fe7-4261-91ec-55499a623fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.375383 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d7e90-9fe7-4261-91ec-55499a623fbf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.384050 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:49.384782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d7e90-9fe7-4261-91ec-55499a623fbf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:49.386589 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d7e90-9fe7-4261-91ec-55499a623fbf None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5b0d7e90-9fe7-4261-91ec-55499a623fbf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:49.404193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:49.405232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:51:49.406182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:49.409338 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 394/794] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 14:51:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:49.415510 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31edab-1e91-4073-93d1-f81a58a6d03b None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.416041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31edab-1e91-4073-93d1-f81a58a6d03b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.416492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31edab-1e91-4073-93d1-f81a58a6d03b None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.418256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 395/795] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:49.419104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31edab-1e91-4073-93d1-f81a58a6d03b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:49.429824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f165a3-849b-4654-88dc-29a9f82027cb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:49.441447 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:49.476260 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cdb03-b4be-4e85-b365-e94b52bf5a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:49.478063 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cdb03-b4be-4e85-b365-e94b52bf5a66 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.478430 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cdb03-b4be-4e85-b365-e94b52bf5a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.479326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cdb03-b4be-4e85-b365-e94b52bf5a66 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.480269 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cdb03-b4be-4e85-b365-e94b52bf5a66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:49.480663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cdb03-b4be-4e85-b365-e94b52bf5a66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:49.482845 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cdb03-b4be-4e85-b365-e94b52bf5a66 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:49.506277 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 401/796] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 14:51:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:51:49.507952 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d68b9-8486-4379-bd69-eca49d66d1cf None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.508345 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d68b9-8486-4379-bd69-eca49d66d1cf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.508700 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d68b9-8486-4379-bd69-eca49d66d1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.529080 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4a9d68b9-8486-4379-bd69-eca49d66d1cf None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:49.530293 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4a9d68b9-8486-4379-bd69-eca49d66d1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:49.543369 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31edab-1e91-4073-93d1-f81a58a6d03b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:49.586841 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b62188-cf89-45c9-92d4-1d54daf70a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:49.587916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b62188-cf89-45c9-92d4-1d54daf70a15 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.588475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b62188-cf89-45c9-92d4-1d54daf70a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.588980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b62188-cf89-45c9-92d4-1d54daf70a15 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.590026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b62188-cf89-45c9-92d4-1d54daf70a15 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:49.590540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b62188-cf89-45c9-92d4-1d54daf70a15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:49.592184 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b62188-cf89-45c9-92d4-1d54daf70a15 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:49.631541 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 402/797] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:51:49.631541 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d68b9-8486-4379-bd69-eca49d66d1cf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:49.702567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:49.754696 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d68b9-8486-4379-bd69-eca49d66d1cf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:49.757412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 396/798] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 14:51:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:49.758464 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b62188-cf89-45c9-92d4-1d54daf70a15 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:49.800052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150498a-26d0-4039-b548-0699bbe2113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:49.801289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150498a-26d0-4039-b548-0699bbe2113f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.801782 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150498a-26d0-4039-b548-0699bbe2113f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.802265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150498a-26d0-4039-b548-0699bbe2113f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.814500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150498a-26d0-4039-b548-0699bbe2113f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:49.816488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150498a-26d0-4039-b548-0699bbe2113f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-0150498a-26d0-4039-b548-0699bbe2113f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:49.837630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:49.838579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:49.839481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:49.839481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:49.839481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:49.839481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:49.841770 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 403/799] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:49.842895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150498a-26d0-4039-b548-0699bbe2113f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:49.852269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:49.878608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b62188-cf89-45c9-92d4-1d54daf70a15 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:49.914682 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2fe29-0821-40c6-9e97-18d05db7e25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:49.915395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2fe29-0821-40c6-9e97-18d05db7e25f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.915680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2fe29-0821-40c6-9e97-18d05db7e25f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.915960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2fe29-0821-40c6-9e97-18d05db7e25f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.926452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2fe29-0821-40c6-9e97-18d05db7e25f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-31b2fe29-0821-40c6-9e97-18d05db7e25f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:49.934049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:49.935021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:49.935890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 397/800] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:49.936634 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b96c6-f249-4373-8fd7-66810480bdba None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:49.937071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b96c6-f249-4373-8fd7-66810480bdba None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:49.937071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b96c6-f249-4373-8fd7-66810480bdba None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:49.938412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 398/801] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:49.939832 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b96c6-f249-4373-8fd7-66810480bdba None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:49.967900 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150498a-26d0-4039-b548-0699bbe2113f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:49.979475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:50.005521 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3f017-bc7f-4d02-8d99-446986d5c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.006289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3f017-bc7f-4d02-8d99-446986d5c323 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.006578 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3f017-bc7f-4d02-8d99-446986d5c323 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.007119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3f017-bc7f-4d02-8d99-446986d5c323 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.054503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b96c6-f249-4373-8fd7-66810480bdba None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:50.082515 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3f017-bc7f-4d02-8d99-446986d5c323 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.084659 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3f017-bc7f-4d02-8d99-446986d5c323 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.093890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfc34a-be0d-4dfc-8148-a1ff33d1523a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.094558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfc34a-be0d-4dfc-8148-a1ff33d1523a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.094844 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfc34a-be0d-4dfc-8148-a1ff33d1523a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.095863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfc34a-be0d-4dfc-8148-a1ff33d1523a None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.096784 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfc34a-be0d-4dfc-8148-a1ff33d1523a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:50.097097 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfc34a-be0d-4dfc-8148-a1ff33d1523a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.098459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfc34a-be0d-4dfc-8148-a1ff33d1523a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.100143 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:50.115149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 399/802] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:50.115149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfc34a-be0d-4dfc-8148-a1ff33d1523a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:50.205520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 404/803] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:51:50.207520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad14eb-098c-4212-a704-d956e7cb75e8 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.207818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad14eb-098c-4212-a704-d956e7cb75e8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.208106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad14eb-098c-4212-a704-d956e7cb75e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.215107 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfc34a-be0d-4dfc-8148-a1ff33d1523a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:50.224818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1dad14eb-098c-4212-a704-d956e7cb75e8 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:50.226138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1dad14eb-098c-4212-a704-d956e7cb75e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:50.229941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:50.252428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87bc2e-10f2-4317-9b81-264c30d3755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.253106 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87bc2e-10f2-4317-9b81-264c30d3755c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.253391 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87bc2e-10f2-4317-9b81-264c30d3755c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.253671 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87bc2e-10f2-4317-9b81-264c30d3755c None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.254418 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87bc2e-10f2-4317-9b81-264c30d3755c None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:50.254726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87bc2e-10f2-4317-9b81-264c30d3755c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.256120 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87bc2e-10f2-4317-9b81-264c30d3755c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.264708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 400/804] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:50.265501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87bc2e-10f2-4317-9b81-264c30d3755c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:50.316303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 405/805] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:51:50.316303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad14eb-098c-4212-a704-d956e7cb75e8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:50.338627 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:50.386690 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87bc2e-10f2-4317-9b81-264c30d3755c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:50.422101 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df3315-c945-4b48-8fca-b7d4569d5ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.422898 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df3315-c945-4b48-8fca-b7d4569d5ade None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.423885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df3315-c945-4b48-8fca-b7d4569d5ade None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.424172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df3315-c945-4b48-8fca-b7d4569d5ade None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.424429 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dad14eb-098c-4212-a704-d956e7cb75e8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:50.431535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df3315-c945-4b48-8fca-b7d4569d5ade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.433218 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df3315-c945-4b48-8fca-b7d4569d5ade None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-66df3315-c945-4b48-8fca-b7d4569d5ade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:50.448322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:50.449452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:50.450574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:50.450574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:50.450574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:50.450574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:50.450574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:50.450574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:50.450574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:50.450574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:50.452685 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 401/806] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:50.453609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df3315-c945-4b48-8fca-b7d4569d5ade None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:50.466046 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb6fb2-3979-475e-bf06-36c9f9e9e867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.466768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb6fb2-3979-475e-bf06-36c9f9e9e867 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.467058 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb6fb2-3979-475e-bf06-36c9f9e9e867 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.467337 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb6fb2-3979-475e-bf06-36c9f9e9e867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.479872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:50.482110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb6fb2-3979-475e-bf06-36c9f9e9e867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.483594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb6fb2-3979-475e-bf06-36c9f9e9e867 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-d2cb6fb2-3979-475e-bf06-36c9f9e9e867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:50.502464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:50.503623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:50.504709 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:50.504709 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:50.504709 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:50.504709 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:50.508507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 406/807] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:50.509403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb6fb2-3979-475e-bf06-36c9f9e9e867 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:50.576828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df3315-c945-4b48-8fca-b7d4569d5ade None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:50.594596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:50.613940 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625faad8-b32d-4589-8c90-6a9ff7758017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.614736 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625faad8-b32d-4589-8c90-6a9ff7758017 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.615021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625faad8-b32d-4589-8c90-6a9ff7758017 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.615305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625faad8-b32d-4589-8c90-6a9ff7758017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.623173 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb6fb2-3979-475e-bf06-36c9f9e9e867 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:50.625390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625faad8-b32d-4589-8c90-6a9ff7758017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.627052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625faad8-b32d-4589-8c90-6a9ff7758017 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-625faad8-b32d-4589-8c90-6a9ff7758017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:50.645274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:50.646211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:50.647131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:50.647131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:50.647131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:50.647131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:50.648873 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 402/808] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:50.649799 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625faad8-b32d-4589-8c90-6a9ff7758017 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:50.664761 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de787b6f-9eb4-4e8b-932c-af9e41850359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.665566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de787b6f-9eb4-4e8b-932c-af9e41850359 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.665855 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de787b6f-9eb4-4e8b-932c-af9e41850359 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.666135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de787b6f-9eb4-4e8b-932c-af9e41850359 None admin] PATH_INFO: `/v3/services/network` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.666798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de787b6f-9eb4-4e8b-932c-af9e41850359 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.668299 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de787b6f-9eb4-4e8b-932c-af9e41850359 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-de787b6f-9eb4-4e8b-932c-af9e41850359 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:50.688363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:50.689424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:51:50.690271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:50.693801 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 407/809] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:50 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:50.694951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de787b6f-9eb4-4e8b-932c-af9e41850359 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:50.757211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625faad8-b32d-4589-8c90-6a9ff7758017 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:50.762724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:50.795948 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893008c-591e-4473-bd83-a048291c2c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.797003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893008c-591e-4473-bd83-a048291c2c0c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.797291 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893008c-591e-4473-bd83-a048291c2c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.797573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893008c-591e-4473-bd83-a048291c2c0c None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.799152 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893008c-591e-4473-bd83-a048291c2c0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:50.799473 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893008c-591e-4473-bd83-a048291c2c0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.800984 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893008c-591e-4473-bd83-a048291c2c0c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.815015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 403/810] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:50.815957 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893008c-591e-4473-bd83-a048291c2c0c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:50.823692 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de787b6f-9eb4-4e8b-932c-af9e41850359 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:50.850275 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:50.863615 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6bb93-eff8-473e-a38a-b6698e1be55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.864308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6bb93-eff8-473e-a38a-b6698e1be55d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.870398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6bb93-eff8-473e-a38a-b6698e1be55d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.870398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6bb93-eff8-473e-a38a-b6698e1be55d None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.870398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6bb93-eff8-473e-a38a-b6698e1be55d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:50.870398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6bb93-eff8-473e-a38a-b6698e1be55d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.870398 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6bb93-eff8-473e-a38a-b6698e1be55d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.928674 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893008c-591e-4473-bd83-a048291c2c0c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:50.965756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc624d-1d83-4709-8fe8-3302bb988c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:50.966438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc624d-1d83-4709-8fe8-3302bb988c27 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:50.966728 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc624d-1d83-4709-8fe8-3302bb988c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:50.967009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc624d-1d83-4709-8fe8-3302bb988c27 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:50.967753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc624d-1d83-4709-8fe8-3302bb988c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:50.968070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc624d-1d83-4709-8fe8-3302bb988c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:50.969535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc624d-1d83-4709-8fe8-3302bb988c27 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:50.983184 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 404/811] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:50.984049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc624d-1d83-4709-8fe8-3302bb988c27 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.001696 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 408/812] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:51.003030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6bb93-eff8-473e-a38a-b6698e1be55d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.075522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:51.087946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc624d-1d83-4709-8fe8-3302bb988c27 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:51.102832 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6bb93-eff8-473e-a38a-b6698e1be55d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:51.119340 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:51.121735 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534371f5-5a04-408d-b86f-6a3199136c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:51.122558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534371f5-5a04-408d-b86f-6a3199136c1d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.123303 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534371f5-5a04-408d-b86f-6a3199136c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.124008 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534371f5-5a04-408d-b86f-6a3199136c1d None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.124754 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534371f5-5a04-408d-b86f-6a3199136c1d None admin] RBAC: Adding query filter params (name=network) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:51.125069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534371f5-5a04-408d-b86f-6a3199136c1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:51.127082 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534371f5-5a04-408d-b86f-6a3199136c1d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:51.131518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 405/813] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:51:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:51.133068 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117d77b-0c71-4247-82ec-982c51e6d81d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.133359 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117d77b-0c71-4247-82ec-982c51e6d81d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.133640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117d77b-0c71-4247-82ec-982c51e6d81d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.134859 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 406/814] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:51.138503 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234bfde-dfc8-45ea-b897-c0e49f35249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:51.139364 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234bfde-dfc8-45ea-b897-c0e49f35249a None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.139488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117d77b-0c71-4247-82ec-982c51e6d81d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.139633 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234bfde-dfc8-45ea-b897-c0e49f35249a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.139907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234bfde-dfc8-45ea-b897-c0e49f35249a None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/bf0dd8c9980e4640aafd34946c4193a3/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.142567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234bfde-dfc8-45ea-b897-c0e49f35249a None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=bf0dd8c9980e4640aafd34946c4193a3, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:51.144102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234bfde-dfc8-45ea-b897-c0e49f35249a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:51.244085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117d77b-0c71-4247-82ec-982c51e6d81d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:51.254639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 409/815] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:51:51 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/bf0dd8c9980e4640aafd34946c4193a3/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:51.255464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234bfde-dfc8-45ea-b897-c0e49f35249a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.284521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085ceb9-c4b7-4ee1-9bbd-494cb02b96ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:51.285478 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ceb9-c4b7-4ee1-9bbd-494cb02b96ca None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.285764 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ceb9-c4b7-4ee1-9bbd-494cb02b96ca None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.286050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ceb9-c4b7-4ee1-9bbd-494cb02b96ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.294709 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085ceb9-c4b7-4ee1-9bbd-494cb02b96ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:51.296503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085ceb9-c4b7-4ee1-9bbd-494cb02b96ca None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c085ceb9-c4b7-4ee1-9bbd-494cb02b96ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:51.315408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:51.316524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 407/816] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:51:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:51.317524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085ceb9-c4b7-4ee1-9bbd-494cb02b96ca None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.335488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:51.378580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234bfde-dfc8-45ea-b897-c0e49f35249a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:51.420106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1fb3d-cc00-4ae5-87d8-8ec5db0480ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:51.421141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1fb3d-cc00-4ae5-87d8-8ec5db0480ff None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.421430 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1fb3d-cc00-4ae5-87d8-8ec5db0480ff None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.421708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1fb3d-cc00-4ae5-87d8-8ec5db0480ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.431387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1fb3d-cc00-4ae5-87d8-8ec5db0480ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:51.432916 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1fb3d-cc00-4ae5-87d8-8ec5db0480ff None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:51.449353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085ceb9-c4b7-4ee1-9bbd-494cb02b96ca None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-0da1fb3d-cc00-4ae5-87d8-8ec5db0480ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:51.449566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:51.450403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:51.451342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:51.451342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:51.451342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:51.451342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:51.451342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:51.451342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:51.451342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:51.451342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:51.451342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 410/817] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:51.452401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86d9cf-cee9-4992-9d3b-5a0f2149c740 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.452401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86d9cf-cee9-4992-9d3b-5a0f2149c740 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.452789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86d9cf-cee9-4992-9d3b-5a0f2149c740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.470261 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4f86d9cf-cee9-4992-9d3b-5a0f2149c740 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:51.471444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4f86d9cf-cee9-4992-9d3b-5a0f2149c740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:51.491069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d436c-065a-464c-bf93-995b09b4c79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:51.491968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d436c-065a-464c-bf93-995b09b4c79d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.492436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d436c-065a-464c-bf93-995b09b4c79d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.492988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d436c-065a-464c-bf93-995b09b4c79d None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.493707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d436c-065a-464c-bf93-995b09b4c79d None admin] RBAC: Adding query filter params (type=network) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:51.494046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d436c-065a-464c-bf93-995b09b4c79d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:51.495411 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d436c-065a-464c-bf93-995b09b4c79d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:51.500952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 408/818] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:51:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:51.502341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d436c-065a-464c-bf93-995b09b4c79d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.510031 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:51.555763 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 411/819] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:51:51.557014 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86d9cf-cee9-4992-9d3b-5a0f2149c740 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.601590 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:51.625603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d436c-065a-464c-bf93-995b09b4c79d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:51.662627 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccbc95-be3e-421e-8d3e-3a592d073853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:51.663293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccbc95-be3e-421e-8d3e-3a592d073853 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.663608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccbc95-be3e-421e-8d3e-3a592d073853 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.663891 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccbc95-be3e-421e-8d3e-3a592d073853 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.664801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccbc95-be3e-421e-8d3e-3a592d073853 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:51.664915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86d9cf-cee9-4992-9d3b-5a0f2149c740 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:51.665597 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccbc95-be3e-421e-8d3e-3a592d073853 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:51.667592 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccbc95-be3e-421e-8d3e-3a592d073853 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:51.686451 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 409/820] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 14:51:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:51.691658 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccbc95-be3e-421e-8d3e-3a592d073853 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.707180 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3351b-ce1f-457b-a499-bfe727a78d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:51.707848 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3351b-ce1f-457b-a499-bfe727a78d93 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.708138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3351b-ce1f-457b-a499-bfe727a78d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.708451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3351b-ce1f-457b-a499-bfe727a78d93 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.709243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3351b-ce1f-457b-a499-bfe727a78d93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:51.709556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3351b-ce1f-457b-a499-bfe727a78d93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:51.710990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3351b-ce1f-457b-a499-bfe727a78d93 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:51.724423 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 412/821] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:51.725453 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3351b-ce1f-457b-a499-bfe727a78d93 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.761462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:51.806980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccbc95-be3e-421e-8d3e-3a592d073853 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:51.842007 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3351b-ce1f-457b-a499-bfe727a78d93 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:51.843211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c0621-dd21-48c2-a5ca-1b5403c1facb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:51.843878 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c0621-dd21-48c2-a5ca-1b5403c1facb None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.844171 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c0621-dd21-48c2-a5ca-1b5403c1facb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.844512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c0621-dd21-48c2-a5ca-1b5403c1facb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.845376 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c0621-dd21-48c2-a5ca-1b5403c1facb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:51.846756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c0621-dd21-48c2-a5ca-1b5403c1facb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:51.850511 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:51.879977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607066cf-7b05-42e9-a3c1-54f7f36755f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:51.880740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607066cf-7b05-42e9-a3c1-54f7f36755f0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:51.881025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607066cf-7b05-42e9-a3c1-54f7f36755f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:51.881300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607066cf-7b05-42e9-a3c1-54f7f36755f0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:51.882430 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607066cf-7b05-42e9-a3c1-54f7f36755f0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:51.883828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607066cf-7b05-42e9-a3c1-54f7f36755f0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-607066cf-7b05-42e9-a3c1-54f7f36755f0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:51.895785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:51.896915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:51:51.897976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:51.900369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 413/822] 10.222.0.88 () {56 vars in 1129 bytes} [Thu Jun 16 14:51:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:51.901411 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607066cf-7b05-42e9-a3c1-54f7f36755f0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:51.974676 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 410/823] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 14:51:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:51.975522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c0621-dd21-48c2-a5ca-1b5403c1facb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:52.011274 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:52.024543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607066cf-7b05-42e9-a3c1-54f7f36755f0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:52.085137 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f73ba-3532-495e-8808-d151aec13c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:52.086159 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f73ba-3532-495e-8808-d151aec13c83 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:52.086553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f73ba-3532-495e-8808-d151aec13c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:52.086936 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f73ba-3532-495e-8808-d151aec13c83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:52.100259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f73ba-3532-495e-8808-d151aec13c83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:52.101983 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f73ba-3532-495e-8808-d151aec13c83 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:52.116527 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c0621-dd21-48c2-a5ca-1b5403c1facb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-836f73ba-3532-495e-8808-d151aec13c83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:52.121737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:52.122890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:52.123998 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:52.123998 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:52.123998 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:52.123998 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:52.125704 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 414/824] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:52.127361 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f73ba-3532-495e-8808-d151aec13c83 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:52.160930 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbcb77-a43b-4d88-b08d-497de8d3c788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:52.161838 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbcb77-a43b-4d88-b08d-497de8d3c788 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:52.162270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbcb77-a43b-4d88-b08d-497de8d3c788 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:52.162710 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbcb77-a43b-4d88-b08d-497de8d3c788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:52.163791 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbcb77-a43b-4d88-b08d-497de8d3c788 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=5368284e42d84e77a6fdb88c2e8dcd3d) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:52.164366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbcb77-a43b-4d88-b08d-497de8d3c788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:52.165999 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbcb77-a43b-4d88-b08d-497de8d3c788 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:52.174779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 411/825] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:51 2022] GET /identity/v3/role_assignments?user.id=5368284e42d84e77a6fdb88c2e8dcd3d&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:52.175833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbcb77-a43b-4d88-b08d-497de8d3c788 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:52.256577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:52.280994 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f73ba-3532-495e-8808-d151aec13c83 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:52.319803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbcb77-a43b-4d88-b08d-497de8d3c788 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:52.329933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7687fe-4ab6-40e3-b6d6-52fe9bed0b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:52.331142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7687fe-4ab6-40e3-b6d6-52fe9bed0b6b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:52.331578 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7687fe-4ab6-40e3-b6d6-52fe9bed0b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:52.332003 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7687fe-4ab6-40e3-b6d6-52fe9bed0b6b None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:52.332928 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7687fe-4ab6-40e3-b6d6-52fe9bed0b6b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:52.333389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7687fe-4ab6-40e3-b6d6-52fe9bed0b6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:52.334974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7687fe-4ab6-40e3-b6d6-52fe9bed0b6b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:52.345489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 415/826] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 14:51:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:52.347491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7687fe-4ab6-40e3-b6d6-52fe9bed0b6b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:52.350540 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:52.365207 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81423c6-edf0-4e6d-809f-ccdcd1cc5a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:52.366224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81423c6-edf0-4e6d-809f-ccdcd1cc5a0d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:52.366558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81423c6-edf0-4e6d-809f-ccdcd1cc5a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:52.366848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81423c6-edf0-4e6d-809f-ccdcd1cc5a0d None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:52.367870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81423c6-edf0-4e6d-809f-ccdcd1cc5a0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:52.368192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81423c6-edf0-4e6d-809f-ccdcd1cc5a0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:52.369782 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81423c6-edf0-4e6d-809f-ccdcd1cc5a0d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:52.390009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 412/827] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:52.396692 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81423c6-edf0-4e6d-809f-ccdcd1cc5a0d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:52.481504 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7687fe-4ab6-40e3-b6d6-52fe9bed0b6b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:52.513384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81423c6-edf0-4e6d-809f-ccdcd1cc5a0d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:52.529234 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0604644-05be-43ed-b909-c0f36ab5b8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:52.529913 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0604644-05be-43ed-b909-c0f36ab5b8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:52.530274 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0604644-05be-43ed-b909-c0f36ab5b8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:52.530570 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0604644-05be-43ed-b909-c0f36ab5b8a9 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:52.531289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0604644-05be-43ed-b909-c0f36ab5b8a9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:52.531604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0604644-05be-43ed-b909-c0f36ab5b8a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:52.532322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:52.534156 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0604644-05be-43ed-b909-c0f36ab5b8a9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:52.539999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 416/828] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 14:51:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:52.545453 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0604644-05be-43ed-b909-c0f36ab5b8a9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:52.556065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e668f6-8ed9-49fd-a2c5-47ef0a25cc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:52.557910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e668f6-8ed9-49fd-a2c5-47ef0a25cc70 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:52.558209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e668f6-8ed9-49fd-a2c5-47ef0a25cc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:52.558493 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e668f6-8ed9-49fd-a2c5-47ef0a25cc70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:52.567642 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e668f6-8ed9-49fd-a2c5-47ef0a25cc70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:52.570370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e668f6-8ed9-49fd-a2c5-47ef0a25cc70 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-75e668f6-8ed9-49fd-a2c5-47ef0a25cc70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:52.590970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:52.591939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:52.592889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:52.592889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:52.592889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:52.592889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:52.592889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:52.592889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:52.592889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:52.592889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:52.592889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 413/829] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:52.599140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e668f6-8ed9-49fd-a2c5-47ef0a25cc70 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:52.634201 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:52.681098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0604644-05be-43ed-b909-c0f36ab5b8a9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:52.719298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa90dc6-b073-4c2a-b381-7f4ab29c7031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:52.719995 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa90dc6-b073-4c2a-b381-7f4ab29c7031 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:52.720313 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa90dc6-b073-4c2a-b381-7f4ab29c7031 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:52.720595 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa90dc6-b073-4c2a-b381-7f4ab29c7031 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:52.721010 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e668f6-8ed9-49fd-a2c5-47ef0a25cc70 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:52.721460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa90dc6-b073-4c2a-b381-7f4ab29c7031 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:52.722707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa90dc6-b073-4c2a-b381-7f4ab29c7031 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:52.757852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140d2fc-1ce4-4030-aa4d-49ede4eee0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:52.758524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140d2fc-1ce4-4030-aa4d-49ede4eee0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:52.758880 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140d2fc-1ce4-4030-aa4d-49ede4eee0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:52.759230 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140d2fc-1ce4-4030-aa4d-49ede4eee0e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:52.760084 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140d2fc-1ce4-4030-aa4d-49ede4eee0e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:52.760698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140d2fc-1ce4-4030-aa4d-49ede4eee0e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:52.761997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140d2fc-1ce4-4030-aa4d-49ede4eee0e5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:52.776324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 414/830] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:52.781895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140d2fc-1ce4-4030-aa4d-49ede4eee0e5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:52.812857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 417/831] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 14:51:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:52.827800 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:52.906275 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140d2fc-1ce4-4030-aa4d-49ede4eee0e5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:52.955644 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7f4c9-e690-4ce9-8772-b06addcfdf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:52.957033 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7f4c9-e690-4ce9-8772-b06addcfdf38 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:52.957531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7f4c9-e690-4ce9-8772-b06addcfdf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:52.958034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7f4c9-e690-4ce9-8772-b06addcfdf38 None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/eadd75d096ac49758fdf012c8d8755ad/roles/b9c2d4c037024096bd7e864cea8b55e8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:52.976426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7f4c9-e690-4ce9-8772-b06addcfdf38 None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=eadd75d096ac49758fdf012c8d8755ad, role_id=b9c2d4c037024096bd7e864cea8b55e8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:52.978223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7f4c9-e690-4ce9-8772-b06addcfdf38 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:53.049885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 415/832] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:51:52 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/eadd75d096ac49758fdf012c8d8755ad/roles/b9c2d4c037024096bd7e864cea8b55e8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:51:53.536565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8e5e4-d9eb-4904-a956-465b120fcaac None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:53.537084 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8e5e4-d9eb-4904-a956-465b120fcaac None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:53.537183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8e5e4-d9eb-4904-a956-465b120fcaac None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:53.539030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 418/833] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:53.544417 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666877f4-0144-4e27-98c0-662b47ec13c8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:53.544710 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666877f4-0144-4e27-98c0-662b47ec13c8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:53.544990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666877f4-0144-4e27-98c0-662b47ec13c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:53.563439 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-666877f4-0144-4e27-98c0-662b47ec13c8 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:53.565300 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-666877f4-0144-4e27-98c0-662b47ec13c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:53.638435 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 416/834] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:51:53.662537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8e5e4-d9eb-4904-a956-465b120fcaac None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:53.777680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8e5e4-d9eb-4904-a956-465b120fcaac None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:53.805850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:53.818968 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf148b-6fab-4999-81fc-9c02d16145f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:53.827960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf148b-6fab-4999-81fc-9c02d16145f1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:53.827960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf148b-6fab-4999-81fc-9c02d16145f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:53.827960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf148b-6fab-4999-81fc-9c02d16145f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:53.834058 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf148b-6fab-4999-81fc-9c02d16145f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e4cf148b-6fab-4999-81fc-9c02d16145f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:53.842036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:53.842978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:51:53.843850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:53.847399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 419/835] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:53.853169 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666877f4-0144-4e27-98c0-662b47ec13c8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:53.866945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:53.963687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666877f4-0144-4e27-98c0-662b47ec13c8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:54.000413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63e2a8-c8de-4de8-ae9d-94a20b7cd365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:54.001256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63e2a8-c8de-4de8-ae9d-94a20b7cd365 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.001610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63e2a8-c8de-4de8-ae9d-94a20b7cd365 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.002053 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63e2a8-c8de-4de8-ae9d-94a20b7cd365 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:54.002810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63e2a8-c8de-4de8-ae9d-94a20b7cd365 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:54.003117 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63e2a8-c8de-4de8-ae9d-94a20b7cd365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:54.004506 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63e2a8-c8de-4de8-ae9d-94a20b7cd365 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:54.018823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 417/836] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:51:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:54.020208 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf148b-6fab-4999-81fc-9c02d16145f1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:54.127394 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf148b-6fab-4999-81fc-9c02d16145f1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:54.163186 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b86004-d1f3-4726-b1c2-f02d457ae26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:54.164034 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b86004-d1f3-4726-b1c2-f02d457ae26d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.164353 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b86004-d1f3-4726-b1c2-f02d457ae26d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.164633 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b86004-d1f3-4726-b1c2-f02d457ae26d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:54.175453 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b86004-d1f3-4726-b1c2-f02d457ae26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:54.176952 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b86004-d1f3-4726-b1c2-f02d457ae26d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-82b86004-d1f3-4726-b1c2-f02d457ae26d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:54.195199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:54.196152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:54.197107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:54.197107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:54.197107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:54.197107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:54.197107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 420/837] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:54.202194 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63e2a8-c8de-4de8-ae9d-94a20b7cd365 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:54.280872 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:54.313989 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63e2a8-c8de-4de8-ae9d-94a20b7cd365 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:54.351413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e002d-384f-4b89-bdf2-339cc5c1ddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:54.352076 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e002d-384f-4b89-bdf2-339cc5c1ddc0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.352391 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e002d-384f-4b89-bdf2-339cc5c1ddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.352668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e002d-384f-4b89-bdf2-339cc5c1ddc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:54.353444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e002d-384f-4b89-bdf2-339cc5c1ddc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:54.353803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e002d-384f-4b89-bdf2-339cc5c1ddc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:54.355165 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e002d-384f-4b89-bdf2-339cc5c1ddc0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:54.372661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 418/838] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:54.374081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b86004-d1f3-4726-b1c2-f02d457ae26d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:54.413528 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:54.481184 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd526c-545d-453c-b020-3140a9ff3071 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.481509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd526c-545d-453c-b020-3140a9ff3071 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.481766 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd526c-545d-453c-b020-3140a9ff3071 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:54.483418 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 419/839] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:54.488205 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b86004-d1f3-4726-b1c2-f02d457ae26d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:54.491203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710b00-44af-4e14-946e-99577fa34b44 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.491719 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710b00-44af-4e14-946e-99577fa34b44 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.492211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710b00-44af-4e14-946e-99577fa34b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:54.513916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-46710b00-44af-4e14-946e-99577fa34b44 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:54.515625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-46710b00-44af-4e14-946e-99577fa34b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:54.528917 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44a971-4c4c-44ab-a5d3-d2f3f69e1f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:54.530065 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44a971-4c4c-44ab-a5d3-d2f3f69e1f39 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.530559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44a971-4c4c-44ab-a5d3-d2f3f69e1f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.531073 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44a971-4c4c-44ab-a5d3-d2f3f69e1f39 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:54.539228 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44a971-4c4c-44ab-a5d3-d2f3f69e1f39 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:54.541411 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44a971-4c4c-44ab-a5d3-d2f3f69e1f39 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-cf44a971-4c4c-44ab-a5d3-d2f3f69e1f39 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:54.556903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:54.558030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:51:54.558990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:54.564446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 421/840] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:51:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:54.565498 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44a971-4c4c-44ab-a5d3-d2f3f69e1f39 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:54.605250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 420/841] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:51:54.628640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46710b00-44af-4e14-946e-99577fa34b44 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:54.671045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:54.678891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44a971-4c4c-44ab-a5d3-d2f3f69e1f39 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:54.721335 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:54.726553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bafb02-073f-4515-b2d2-d5ece9fcab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:54.727630 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bafb02-073f-4515-b2d2-d5ece9fcab8e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.728115 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bafb02-073f-4515-b2d2-d5ece9fcab8e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.728609 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bafb02-073f-4515-b2d2-d5ece9fcab8e None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:54.729649 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bafb02-073f-4515-b2d2-d5ece9fcab8e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:54.730623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bafb02-073f-4515-b2d2-d5ece9fcab8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:54.732641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bafb02-073f-4515-b2d2-d5ece9fcab8e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:54.759684 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46710b00-44af-4e14-946e-99577fa34b44 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:54.761212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 422/842] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 14:51:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:54.771640 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bafb02-073f-4515-b2d2-d5ece9fcab8e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:54.814841 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa06db-85b3-42a7-ad1e-1af94c4ec158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:54.815669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa06db-85b3-42a7-ad1e-1af94c4ec158 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.815955 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa06db-85b3-42a7-ad1e-1af94c4ec158 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.816250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa06db-85b3-42a7-ad1e-1af94c4ec158 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:54.816922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa06db-85b3-42a7-ad1e-1af94c4ec158 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:54.818307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa06db-85b3-42a7-ad1e-1af94c4ec158 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7daa06db-85b3-42a7-ad1e-1af94c4ec158 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:54.833253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:54.834311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:51:54.835217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:54.840403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 421/843] 10.222.0.88 () {56 vars in 1132 bytes} [Thu Jun 16 14:51:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:51:54.841885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa06db-85b3-42a7-ad1e-1af94c4ec158 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:54.894331 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bafb02-073f-4515-b2d2-d5ece9fcab8e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:54.933378 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013fca-9ac0-4259-8c0b-a40521c3aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:54.934347 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013fca-9ac0-4259-8c0b-a40521c3aa26 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.934898 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013fca-9ac0-4259-8c0b-a40521c3aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.935387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013fca-9ac0-4259-8c0b-a40521c3aa26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:54.960108 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa06db-85b3-42a7-ad1e-1af94c4ec158 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:54.962431 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:54.988620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:54.998206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead3e13-24ef-46fa-96c9-15fd3a4da3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:54.998862 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead3e13-24ef-46fa-96c9-15fd3a4da3ca None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:54.999149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead3e13-24ef-46fa-96c9-15fd3a4da3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:54.999429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead3e13-24ef-46fa-96c9-15fd3a4da3ca None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.000137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead3e13-24ef-46fa-96c9-15fd3a4da3ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:55.000491 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead3e13-24ef-46fa-96c9-15fd3a4da3ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:55.002951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead3e13-24ef-46fa-96c9-15fd3a4da3ca None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:55.014089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 422/844] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:51:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:55.015431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead3e13-24ef-46fa-96c9-15fd3a4da3ca None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:55.017225 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2013fca-9ac0-4259-8c0b-a40521c3aa26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:55.020497 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2013fca-9ac0-4259-8c0b-a40521c3aa26 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:55.131895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead3e13-24ef-46fa-96c9-15fd3a4da3ca None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:55.139295 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 423/845] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:51:55.140669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013fca-9ac0-4259-8c0b-a40521c3aa26 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:55.176881 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05062ee-2340-4dee-832a-a5f54c404971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:55.177506 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05062ee-2340-4dee-832a-a5f54c404971 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.177774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05062ee-2340-4dee-832a-a5f54c404971 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.178053 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05062ee-2340-4dee-832a-a5f54c404971 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.178777 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05062ee-2340-4dee-832a-a5f54c404971 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:55.179084 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05062ee-2340-4dee-832a-a5f54c404971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:55.180624 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05062ee-2340-4dee-832a-a5f54c404971 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:55.192269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 423/846] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:55.208070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:55.226353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c59e5e-8cf4-4bfb-9f36-ccf9d566fbc7 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.226696 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c59e5e-8cf4-4bfb-9f36-ccf9d566fbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.226910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c59e5e-8cf4-4bfb-9f36-ccf9d566fbc7 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.228663 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 424/847] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:51:55.234245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ffd5-0887-4802-8de7-cf4b3e13c533 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.234570 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ffd5-0887-4802-8de7-cf4b3e13c533 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.234808 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ffd5-0887-4802-8de7-cf4b3e13c533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.253727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cdd3ffd5-0887-4802-8de7-cf4b3e13c533 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:55.254961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cdd3ffd5-0887-4802-8de7-cf4b3e13c533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:55.268269 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013fca-9ac0-4259-8c0b-a40521c3aa26 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:55.314594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40555c8-768a-4755-914b-bf74dcfa8210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:55.316078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40555c8-768a-4755-914b-bf74dcfa8210 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.316654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40555c8-768a-4755-914b-bf74dcfa8210 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.317147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40555c8-768a-4755-914b-bf74dcfa8210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.318908 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:55.331292 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40555c8-768a-4755-914b-bf74dcfa8210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:55.333200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40555c8-768a-4755-914b-bf74dcfa8210 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:55.346725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 425/848] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a40555c8-768a-4755-914b-bf74dcfa8210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:55.353894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:55.354874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:55.355800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:55.355800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:55.355800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:55.355800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:55.358175 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 424/849] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:55.364292 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40555c8-768a-4755-914b-bf74dcfa8210 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:55.365157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8e3a4-9bd6-477c-91f1-1470623dba13 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.365410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8e3a4-9bd6-477c-91f1-1470623dba13 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.365695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8e3a4-9bd6-477c-91f1-1470623dba13 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.367354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 426/850] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:55.368112 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8e3a4-9bd6-477c-91f1-1470623dba13 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:55.482815 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40555c8-768a-4755-914b-bf74dcfa8210 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:55.486702 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:55.496135 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8e3a4-9bd6-477c-91f1-1470623dba13 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:55.526655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e67f18-22bd-42c1-8e36-07a2fd3b333d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:55.527589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e67f18-22bd-42c1-8e36-07a2fd3b333d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.527887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e67f18-22bd-42c1-8e36-07a2fd3b333d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.528162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e67f18-22bd-42c1-8e36-07a2fd3b333d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.528864 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e67f18-22bd-42c1-8e36-07a2fd3b333d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:55.530277 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e67f18-22bd-42c1-8e36-07a2fd3b333d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:55.540147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e97e8-613b-4e03-a8d1-b78bac385b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:55.541103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e97e8-613b-4e03-a8d1-b78bac385b4f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.541569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e97e8-613b-4e03-a8d1-b78bac385b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.542028 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e97e8-613b-4e03-a8d1-b78bac385b4f None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.542965 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e97e8-613b-4e03-a8d1-b78bac385b4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c8e67f18-22bd-42c1-8e36-07a2fd3b333d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:55.543085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:55.544369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:55.547501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 425/851] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:55.548879 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e97e8-613b-4e03-a8d1-b78bac385b4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:55.549038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5df7c9-8ea1-40b4-9f0f-39a13cd1b7c0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.549112 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e97e8-613b-4e03-a8d1-b78bac385b4f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:55.549182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5df7c9-8ea1-40b4-9f0f-39a13cd1b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.549182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5df7c9-8ea1-40b4-9f0f-39a13cd1b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.563244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 427/852] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:55.564994 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e97e8-613b-4e03-a8d1-b78bac385b4f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:55.566392 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0e5df7c9-8ea1-40b4-9f0f-39a13cd1b7c0 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:55.567617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0e5df7c9-8ea1-40b4-9f0f-39a13cd1b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:55.616752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:55.636716 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 426/853] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:51:55.637782 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5df7c9-8ea1-40b4-9f0f-39a13cd1b7c0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:55.684901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e97e8-613b-4e03-a8d1-b78bac385b4f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:55.719423 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad700acc-1177-43f1-9718-f1de71f366ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:55.720806 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad700acc-1177-43f1-9718-f1de71f366ac None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.721096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad700acc-1177-43f1-9718-f1de71f366ac None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.721379 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad700acc-1177-43f1-9718-f1de71f366ac None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.722093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad700acc-1177-43f1-9718-f1de71f366ac None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:55.722405 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad700acc-1177-43f1-9718-f1de71f366ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:55.723790 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad700acc-1177-43f1-9718-f1de71f366ac None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:55.733402 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 428/854] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:55.733402 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad700acc-1177-43f1-9718-f1de71f366ac None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:55.755538 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:55.761503 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5df7c9-8ea1-40b4-9f0f-39a13cd1b7c0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:55.799620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49700fcd-4ca0-44a8-91da-a2b8221bb2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:55.800441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49700fcd-4ca0-44a8-91da-a2b8221bb2bb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.800731 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49700fcd-4ca0-44a8-91da-a2b8221bb2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.801012 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49700fcd-4ca0-44a8-91da-a2b8221bb2bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.807883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49700fcd-4ca0-44a8-91da-a2b8221bb2bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:55.809379 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49700fcd-4ca0-44a8-91da-a2b8221bb2bb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-49700fcd-4ca0-44a8-91da-a2b8221bb2bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:55.823854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:55.825028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 427/855] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:55.825959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49700fcd-4ca0-44a8-91da-a2b8221bb2bb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:55.839875 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad700acc-1177-43f1-9718-f1de71f366ac None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:55.850653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:55.878813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e777fc6-335c-4141-bc8c-e18310b2d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:55.879614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e777fc6-335c-4141-bc8c-e18310b2d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.879898 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e777fc6-335c-4141-bc8c-e18310b2d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.880177 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e777fc6-335c-4141-bc8c-e18310b2d6f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.892529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e777fc6-335c-4141-bc8c-e18310b2d6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2e777fc6-335c-4141-bc8c-e18310b2d6f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:51:55.900427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:55.901395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:55.902280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 429/856] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:55.902986 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e777fc6-335c-4141-bc8c-e18310b2d6f2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:55.942487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49700fcd-4ca0-44a8-91da-a2b8221bb2bb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:55.977229 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76fa0-816c-4ff6-a35d-325afeb201f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:55.977897 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76fa0-816c-4ff6-a35d-325afeb201f6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:55.978183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76fa0-816c-4ff6-a35d-325afeb201f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:55.978462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76fa0-816c-4ff6-a35d-325afeb201f6 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:55.979171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc76fa0-816c-4ff6-a35d-325afeb201f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:55.979486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc76fa0-816c-4ff6-a35d-325afeb201f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:55.980896 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc76fa0-816c-4ff6-a35d-325afeb201f6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:55.986099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 428/857] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:55.986828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76fa0-816c-4ff6-a35d-325afeb201f6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:56.022498 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:56.028714 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e777fc6-335c-4141-bc8c-e18310b2d6f2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:56.075588 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ac2c9-02d7-445b-a75f-df337bcd2f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:56.076301 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ac2c9-02d7-445b-a75f-df337bcd2f91 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:56.076596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ac2c9-02d7-445b-a75f-df337bcd2f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:56.076885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ac2c9-02d7-445b-a75f-df337bcd2f91 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:56.077695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ac2c9-02d7-445b-a75f-df337bcd2f91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:56.078014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ac2c9-02d7-445b-a75f-df337bcd2f91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:56.080205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ac2c9-02d7-445b-a75f-df337bcd2f91 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:56.100396 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 430/858] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:56.104728 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ac2c9-02d7-445b-a75f-df337bcd2f91 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:56.121475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76fa0-816c-4ff6-a35d-325afeb201f6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:56.136621 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:56.161112 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32a5b7-8cb7-4f54-b2a1-10eca368fe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:56.161800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32a5b7-8cb7-4f54-b2a1-10eca368fe2c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:56.162092 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32a5b7-8cb7-4f54-b2a1-10eca368fe2c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:56.162372 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32a5b7-8cb7-4f54-b2a1-10eca368fe2c None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:56.163175 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32a5b7-8cb7-4f54-b2a1-10eca368fe2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:56.163439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32a5b7-8cb7-4f54-b2a1-10eca368fe2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:56.164883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32a5b7-8cb7-4f54-b2a1-10eca368fe2c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:56.179199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 429/859] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:51:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:56.180119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32a5b7-8cb7-4f54-b2a1-10eca368fe2c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:56.234781 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ac2c9-02d7-445b-a75f-df337bcd2f91 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:56.278896 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:56.280901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f38fc-30a9-4ee6-bb53-fe7c7650f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:56.281569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f38fc-30a9-4ee6-bb53-fe7c7650f385 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:56.281858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f38fc-30a9-4ee6-bb53-fe7c7650f385 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:56.282324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f38fc-30a9-4ee6-bb53-fe7c7650f385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:56.283185 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f38fc-30a9-4ee6-bb53-fe7c7650f385 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=bf0dd8c9980e4640aafd34946c4193a3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:56.283496 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f38fc-30a9-4ee6-bb53-fe7c7650f385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:56.285577 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f38fc-30a9-4ee6-bb53-fe7c7650f385 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:56.293623 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 431/860] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:56 2022] GET /identity/v3/role_assignments?user.id=bf0dd8c9980e4640aafd34946c4193a3&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:56.301154 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32a5b7-8cb7-4f54-b2a1-10eca368fe2c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:56.341591 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c3587-ffca-4a08-ae22-f576f58bd1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:56.342715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c3587-ffca-4a08-ae22-f576f58bd1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:56.343305 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c3587-ffca-4a08-ae22-f576f58bd1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:56.343772 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c3587-ffca-4a08-ae22-f576f58bd1b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:56.357242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c3587-ffca-4a08-ae22-f576f58bd1b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:56.359311 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78c3587-ffca-4a08-ae22-f576f58bd1b5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f78c3587-ffca-4a08-ae22-f576f58bd1b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:56.380951 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:56.382187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:56.383105 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:56.383105 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:56.383105 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:56.383105 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:56.385249 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 430/861] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:56.395950 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f38fc-30a9-4ee6-bb53-fe7c7650f385 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:56.502600 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:56.517251 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f38fc-30a9-4ee6-bb53-fe7c7650f385 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:56.563968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbac1d0-3d58-4163-8487-0e8c98e7611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:56.564576 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbac1d0-3d58-4163-8487-0e8c98e7611c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:56.564864 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbac1d0-3d58-4163-8487-0e8c98e7611c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:56.565150 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbac1d0-3d58-4163-8487-0e8c98e7611c None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:56.565877 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbac1d0-3d58-4163-8487-0e8c98e7611c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:56.566190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbac1d0-3d58-4163-8487-0e8c98e7611c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:56.567574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbac1d0-3d58-4163-8487-0e8c98e7611c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:56.580290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 432/862] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:56.585500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c3587-ffca-4a08-ae22-f576f58bd1b5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:56.692830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c3587-ffca-4a08-ae22-f576f58bd1b5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:56.729910 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85230fc-b9f1-43db-816d-57e3c1345801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:56.730732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85230fc-b9f1-43db-816d-57e3c1345801 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:56.731024 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85230fc-b9f1-43db-816d-57e3c1345801 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:56.731305 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85230fc-b9f1-43db-816d-57e3c1345801 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:56.739880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85230fc-b9f1-43db-816d-57e3c1345801 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:56.741483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85230fc-b9f1-43db-816d-57e3c1345801 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:56.742337 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f85230fc-b9f1-43db-816d-57e3c1345801 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:51:56.757672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:56.758655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:56.759614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 431/863] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:51:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:56.764509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbac1d0-3d58-4163-8487-0e8c98e7611c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:56.872537 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbac1d0-3d58-4163-8487-0e8c98e7611c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:56.909696 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec10dc-3f72-4658-9901-eb0b4d1dd2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:56.910729 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec10dc-3f72-4658-9901-eb0b4d1dd2eb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:56.911244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec10dc-3f72-4658-9901-eb0b4d1dd2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:56.911738 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec10dc-3f72-4658-9901-eb0b4d1dd2eb None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:56.912816 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec10dc-3f72-4658-9901-eb0b4d1dd2eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:56.913334 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec10dc-3f72-4658-9901-eb0b4d1dd2eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:56.914973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec10dc-3f72-4658-9901-eb0b4d1dd2eb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:56.933650 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 433/864] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 14:51:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:56.940267 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85230fc-b9f1-43db-816d-57e3c1345801 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:57.051096 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85230fc-b9f1-43db-816d-57e3c1345801 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:57.087311 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a948-81dd-44e1-9e7e-05835f4e0f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:57.087958 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a948-81dd-44e1-9e7e-05835f4e0f61 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:57.088266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a948-81dd-44e1-9e7e-05835f4e0f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:57.088549 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a948-81dd-44e1-9e7e-05835f4e0f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:57.159643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:57.163074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a948-81dd-44e1-9e7e-05835f4e0f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:57.164551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a948-81dd-44e1-9e7e-05835f4e0f61 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:57.273663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 432/865] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:51:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:51:57.278747 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec10dc-3f72-4658-9901-eb0b4d1dd2eb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:57.304574 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:57.385076 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec10dc-3f72-4658-9901-eb0b4d1dd2eb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:57.421773 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0e7a9-1a13-4ab5-b74e-27c247d19f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:57.422487 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0e7a9-1a13-4ab5-b74e-27c247d19f6c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:57.423391 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0e7a9-1a13-4ab5-b74e-27c247d19f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:57.423675 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0e7a9-1a13-4ab5-b74e-27c247d19f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:57.434466 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0e7a9-1a13-4ab5-b74e-27c247d19f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:57.435977 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0e7a9-1a13-4ab5-b74e-27c247d19f6c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d6e0e7a9-1a13-4ab5-b74e-27c247d19f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:51:57.453909 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:57.455009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:57.455914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:51:57.455914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:51:57.455914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:51:57.455914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:51:57.455914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 434/866] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:57.461132 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a948-81dd-44e1-9e7e-05835f4e0f61 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:57.563053 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7320-808e-4612-9186-a7982de00ed3 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:57.563508 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7320-808e-4612-9186-a7982de00ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:57.563745 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7320-808e-4612-9186-a7982de00ed3 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:57.565395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 435/867] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:57.570516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468b5d-44fd-459a-8b94-ae0c84c56388 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:57.570810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468b5d-44fd-459a-8b94-ae0c84c56388 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:57.571096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468b5d-44fd-459a-8b94-ae0c84c56388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:57.583582 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:57.590611 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a948-81dd-44e1-9e7e-05835f4e0f61 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:57.591177 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-96468b5d-44fd-459a-8b94-ae0c84c56388 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:57.592113 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-96468b5d-44fd-459a-8b94-ae0c84c56388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:57.627598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313ab9f-cab2-4301-915f-20e6eb5c0d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:57.628345 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313ab9f-cab2-4301-915f-20e6eb5c0d5e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:57.628667 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313ab9f-cab2-4301-915f-20e6eb5c0d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:57.628952 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313ab9f-cab2-4301-915f-20e6eb5c0d5e None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:57.629745 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313ab9f-cab2-4301-915f-20e6eb5c0d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:57.630124 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313ab9f-cab2-4301-915f-20e6eb5c0d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:57.631796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f313ab9f-cab2-4301-915f-20e6eb5c0d5e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:57.644954 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 433/868] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:57.651245 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313ab9f-cab2-4301-915f-20e6eb5c0d5e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:57.670970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:57.692721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 436/869] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:51:57.710366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96468b5d-44fd-459a-8b94-ae0c84c56388 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:57.772374 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313ab9f-cab2-4301-915f-20e6eb5c0d5e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:57.812744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea976ab-443a-47d1-9501-e3eaed6982d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:57.813588 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea976ab-443a-47d1-9501-e3eaed6982d8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:57.813877 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea976ab-443a-47d1-9501-e3eaed6982d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:57.814150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea976ab-443a-47d1-9501-e3eaed6982d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:57.824590 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea976ab-443a-47d1-9501-e3eaed6982d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:57.826057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea976ab-443a-47d1-9501-e3eaed6982d8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:57.840271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96468b5d-44fd-459a-8b94-ae0c84c56388 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fea976ab-443a-47d1-9501-e3eaed6982d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:51:57.846319 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:57.847617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:57.849537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:51:57.849537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:51:57.849537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:51:57.849537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:51:57.849537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:51:57.849537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:51:57.849537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:51:57.849537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:57.856251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 434/870] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:51:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:57.856251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea976ab-443a-47d1-9501-e3eaed6982d8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:57.881326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:57.883562 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186deff-6165-46a0-ab13-883fe9b1379f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:57.884366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186deff-6165-46a0-ab13-883fe9b1379f None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:57.884798 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186deff-6165-46a0-ab13-883fe9b1379f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:57.885197 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186deff-6165-46a0-ab13-883fe9b1379f None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:57.886050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186deff-6165-46a0-ab13-883fe9b1379f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:57.887524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186deff-6165-46a0-ab13-883fe9b1379f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:57.974171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea976ab-443a-47d1-9501-e3eaed6982d8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:57.986928 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 437/871] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:51:57 2022] POST /identity/v3/services => generated 269 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:51:58.025481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d525e52-3b6f-4e04-b3ea-181a03bc3c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:58.026418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d525e52-3b6f-4e04-b3ea-181a03bc3c77 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:58.026874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d525e52-3b6f-4e04-b3ea-181a03bc3c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:58.027303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d525e52-3b6f-4e04-b3ea-181a03bc3c77 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:58.028348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d525e52-3b6f-4e04-b3ea-181a03bc3c77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:58.028830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d525e52-3b6f-4e04-b3ea-181a03bc3c77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:58.030441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d525e52-3b6f-4e04-b3ea-181a03bc3c77 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:58.045120 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 435/872] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:51:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:58.051698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186deff-6165-46a0-ab13-883fe9b1379f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:58.116104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:58.172601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186deff-6165-46a0-ab13-883fe9b1379f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:58.211169 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c7269-7030-4135-8007-5b7769d72866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:58.212077 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c7269-7030-4135-8007-5b7769d72866 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:58.212542 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c7269-7030-4135-8007-5b7769d72866 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:58.212989 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c7269-7030-4135-8007-5b7769d72866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:58.214002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c7269-7030-4135-8007-5b7769d72866 None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=eadd75d096ac49758fdf012c8d8755ad) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:58.214477 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c7269-7030-4135-8007-5b7769d72866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:58.216069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c7269-7030-4135-8007-5b7769d72866 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:58.223233 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 438/873] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:51:58 2022] GET /identity/v3/role_assignments?user.id=eadd75d096ac49758fdf012c8d8755ad&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:58.319857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63075e-d8bf-4f08-ace5-565c586b140f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:58.320186 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63075e-d8bf-4f08-ace5-565c586b140f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:58.320473 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63075e-d8bf-4f08-ace5-565c586b140f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:58.322237 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 436/874] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:58.328044 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207812e0-4ce9-4a92-b92c-67150bffa797 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:58.328617 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207812e0-4ce9-4a92-b92c-67150bffa797 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:58.328733 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207812e0-4ce9-4a92-b92c-67150bffa797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:58.349024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-207812e0-4ce9-4a92-b92c-67150bffa797 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:58.350231 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-207812e0-4ce9-4a92-b92c-67150bffa797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:58.431437 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 439/875] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:58 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:51:58.455374 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63075e-d8bf-4f08-ace5-565c586b140f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:58.542477 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:58.580449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63075e-d8bf-4f08-ace5-565c586b140f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:58.617948 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fc1ae-95c3-4a22-bf5b-6bf356349573 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:58.618274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fc1ae-95c3-4a22-bf5b-6bf356349573 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:58.618525 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fc1ae-95c3-4a22-bf5b-6bf356349573 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:58.618644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b37ff6-133f-4e9d-a577-fab2150bd1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:58.619188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b37ff6-133f-4e9d-a577-fab2150bd1b2 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:58.619477 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b37ff6-133f-4e9d-a577-fab2150bd1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:58.619760 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b37ff6-133f-4e9d-a577-fab2150bd1b2 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:58.620249 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 440/876] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:51:58.620598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b37ff6-133f-4e9d-a577-fab2150bd1b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:58.622043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b37ff6-133f-4e9d-a577-fab2150bd1b2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:58.625349 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe9433-5151-4dcd-b53b-0b1840552565 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:58.626279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe9433-5151-4dcd-b53b-0b1840552565 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:58.626748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe9433-5151-4dcd-b53b-0b1840552565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:58.643946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7bbe9433-5151-4dcd-b53b-0b1840552565 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:58.645176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7bbe9433-5151-4dcd-b53b-0b1840552565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:58.691528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:58.720427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 437/877] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:51:58 2022] POST /identity/v3/services => generated 251 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:51:58.737544 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 441/878] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:51:58.763513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b37ff6-133f-4e9d-a577-fab2150bd1b2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:58.885719 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b37ff6-133f-4e9d-a577-fab2150bd1b2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:58.917407 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:58.922477 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad6c13-025a-4fc7-b181-cd8c89cffdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:58.924040 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad6c13-025a-4fc7-b181-cd8c89cffdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:58.924412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad6c13-025a-4fc7-b181-cd8c89cffdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:58.924741 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad6c13-025a-4fc7-b181-cd8c89cffdb8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:58.925450 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad6c13-025a-4fc7-b181-cd8c89cffdb8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:58.927009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad6c13-025a-4fc7-b181-cd8c89cffdb8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-43ad6c13-025a-4fc7-b181-cd8c89cffdb8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:51:58.940682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:51:58.941721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:51:58.943004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 438/879] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:51:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:51:58.948453 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe9433-5151-4dcd-b53b-0b1840552565 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:59.060029 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe9433-5151-4dcd-b53b-0b1840552565 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:59.098044 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c2a4b-68b7-4dec-b850-2d1b0531ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:59.098715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c2a4b-68b7-4dec-b850-2d1b0531ce61 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:59.099006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c2a4b-68b7-4dec-b850-2d1b0531ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:59.099288 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c2a4b-68b7-4dec-b850-2d1b0531ce61 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:59.100030 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c2a4b-68b7-4dec-b850-2d1b0531ce61 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:59.100374 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c2a4b-68b7-4dec-b850-2d1b0531ce61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:59.101917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c2a4b-68b7-4dec-b850-2d1b0531ce61 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:59.107290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 442/880] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:51:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:51:59.113157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad6c13-025a-4fc7-b181-cd8c89cffdb8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:59.218905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad6c13-025a-4fc7-b181-cd8c89cffdb8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:59.257414 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33969e-21b8-443a-abe8-57fb17382660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:59.258532 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33969e-21b8-443a-abe8-57fb17382660 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:59.259039 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33969e-21b8-443a-abe8-57fb17382660 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:59.259551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33969e-21b8-443a-abe8-57fb17382660 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:59.260513 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33969e-21b8-443a-abe8-57fb17382660 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:51:59.261048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33969e-21b8-443a-abe8-57fb17382660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:59.262661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33969e-21b8-443a-abe8-57fb17382660 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:59.276369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 439/881] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:51:59 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:00.326009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c05ae-2a77-45f0-8c32-c12074ad604a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:00.326781 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c05ae-2a77-45f0-8c32-c12074ad604a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:00.326781 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c05ae-2a77-45f0-8c32-c12074ad604a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:00.328371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 443/882] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:00.333817 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba344d8-476c-491f-9639-a89d614c943d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:00.334257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba344d8-476c-491f-9639-a89d614c943d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:00.334461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba344d8-476c-491f-9639-a89d614c943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:00.352464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5ba344d8-476c-491f-9639-a89d614c943d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:00.353643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5ba344d8-476c-491f-9639-a89d614c943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:00.419911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:00.432610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 440/883] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:52:00.456865 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c05ae-2a77-45f0-8c32-c12074ad604a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:00.507221 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:00.540187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fb363-f4c8-4e1a-b741-c784ad262974 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:00.540530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fb363-f4c8-4e1a-b741-c784ad262974 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:00.540828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fb363-f4c8-4e1a-b741-c784ad262974 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:00.542456 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 441/884] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:52:00.547413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b57d18-9fe4-4f58-acc2-31c8a92705e9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:00.547886 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b57d18-9fe4-4f58-acc2-31c8a92705e9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:00.548178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b57d18-9fe4-4f58-acc2-31c8a92705e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:00.564847 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c9b57d18-9fe4-4f58-acc2-31c8a92705e9 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:00.566041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c9b57d18-9fe4-4f58-acc2-31c8a92705e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:00.586249 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c05ae-2a77-45f0-8c32-c12074ad604a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:00.627910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa54e9-c3d5-460e-bd18-5ddc8f058b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:00.628756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa54e9-c3d5-460e-bd18-5ddc8f058b8f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:00.629046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa54e9-c3d5-460e-bd18-5ddc8f058b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:00.629328 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa54e9-c3d5-460e-bd18-5ddc8f058b8f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:00.630061 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa54e9-c3d5-460e-bd18-5ddc8f058b8f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:00.631687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa54e9-c3d5-460e-bd18-5ddc8f058b8f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:00.642771 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 442/885] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-41fa54e9-c3d5-460e-bd18-5ddc8f058b8f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:00.647973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:00.649081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:00.650024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 444/886] 10.222.0.88 () {56 vars in 1132 bytes} [Thu Jun 16 14:52:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:00.658114 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b57d18-9fe4-4f58-acc2-31c8a92705e9 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:00.668807 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa54e9-c3d5-460e-bd18-5ddc8f058b8f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:00.774539 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b57d18-9fe4-4f58-acc2-31c8a92705e9 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:00.795491 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:00.797018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:00.799351 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa54e9-c3d5-460e-bd18-5ddc8f058b8f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:00.814641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155851c-fd18-419a-8c4f-f2b8d2fd6107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:00.815588 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155851c-fd18-419a-8c4f-f2b8d2fd6107 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:00.815879 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155851c-fd18-419a-8c4f-f2b8d2fd6107 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:00.816340 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155851c-fd18-419a-8c4f-f2b8d2fd6107 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:00.817359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155851c-fd18-419a-8c4f-f2b8d2fd6107 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:00.817678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155851c-fd18-419a-8c4f-f2b8d2fd6107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:00.819062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155851c-fd18-419a-8c4f-f2b8d2fd6107 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:00.828396 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 443/887] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:52:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:00.831880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155851c-fd18-419a-8c4f-f2b8d2fd6107 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:00.850145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545cd60-db1f-4e08-8b9d-1eba2df5f468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:00.850145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545cd60-db1f-4e08-8b9d-1eba2df5f468 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:00.850145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545cd60-db1f-4e08-8b9d-1eba2df5f468 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:00.850145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545cd60-db1f-4e08-8b9d-1eba2df5f468 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:00.855260 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545cd60-db1f-4e08-8b9d-1eba2df5f468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3545cd60-db1f-4e08-8b9d-1eba2df5f468 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:00.864246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:00.866540 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:00.868653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:00.872311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 445/888] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 14:52:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:00.872311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545cd60-db1f-4e08-8b9d-1eba2df5f468 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:00.960218 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155851c-fd18-419a-8c4f-f2b8d2fd6107 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.000375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545cd60-db1f-4e08-8b9d-1eba2df5f468 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.001237 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add61ea-28b5-46d7-96e2-894356d7d885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.002275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add61ea-28b5-46d7-96e2-894356d7d885 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.002561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add61ea-28b5-46d7-96e2-894356d7d885 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.002843 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add61ea-28b5-46d7-96e2-894356d7d885 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.003575 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add61ea-28b5-46d7-96e2-894356d7d885 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:01.003883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add61ea-28b5-46d7-96e2-894356d7d885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.005336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add61ea-28b5-46d7-96e2-894356d7d885 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.017683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 444/889] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:52:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:01.018784 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add61ea-28b5-46d7-96e2-894356d7d885 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.045613 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4c7f4-4234-4057-b208-b7e6f2b2ad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.046366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4c7f4-4234-4057-b208-b7e6f2b2ad8c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.046652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4c7f4-4234-4057-b208-b7e6f2b2ad8c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.046980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4c7f4-4234-4057-b208-b7e6f2b2ad8c None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.048427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4c7f4-4234-4057-b208-b7e6f2b2ad8c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:01.048745 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4c7f4-4234-4057-b208-b7e6f2b2ad8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.050182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4c7f4-4234-4057-b208-b7e6f2b2ad8c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.059712 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 446/890] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 14:52:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:01.061016 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5347f-6126-43cc-a690-e5ff95035404 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.061303 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5347f-6126-43cc-a690-e5ff95035404 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.061640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5347f-6126-43cc-a690-e5ff95035404 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.063531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 447/891] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:01.068590 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5347f-6126-43cc-a690-e5ff95035404 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.085064 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:01.096006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:01.138326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add61ea-28b5-46d7-96e2-894356d7d885 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.175921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd583b9b-e28e-448c-be03-e982c68de151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.176981 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd583b9b-e28e-448c-be03-e982c68de151 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.177511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd583b9b-e28e-448c-be03-e982c68de151 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.177896 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd583b9b-e28e-448c-be03-e982c68de151 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.178696 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd583b9b-e28e-448c-be03-e982c68de151 None admin] RBAC: Adding query filter params (interface=public, service_id=ca0a413ba20c4dc3bcdd109be0ee2d4f, region_id=RegionOne) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:01.179014 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd583b9b-e28e-448c-be03-e982c68de151 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.180421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd583b9b-e28e-448c-be03-e982c68de151 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.181238 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5347f-6126-43cc-a690-e5ff95035404 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.192006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 445/892] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/endpoints?service_id=ca0a413ba20c4dc3bcdd109be0ee2d4f&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:01.192006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c2462-fefc-4402-9f30-d17b83f17210 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.192006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c2462-fefc-4402-9f30-d17b83f17210 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.192006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c2462-fefc-4402-9f30-d17b83f17210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.209373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e01c2462-fefc-4402-9f30-d17b83f17210 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:01.210581 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e01c2462-fefc-4402-9f30-d17b83f17210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:01.219112 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10275759-1b51-424e-b500-2a84228cd6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.221447 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10275759-1b51-424e-b500-2a84228cd6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.221737 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10275759-1b51-424e-b500-2a84228cd6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.222015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10275759-1b51-424e-b500-2a84228cd6b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.233725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10275759-1b51-424e-b500-2a84228cd6b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.235969 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10275759-1b51-424e-b500-2a84228cd6b2 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-10275759-1b51-424e-b500-2a84228cd6b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:01.254181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:01.255179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:01.256079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:01.256079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:01.256079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:01.256079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:01.256079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 448/893] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:01.257150 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10275759-1b51-424e-b500-2a84228cd6b2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.287202 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 446/894] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:01.288408 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c2462-fefc-4402-9f30-d17b83f17210 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.337771 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:01.354166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:01.382272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10275759-1b51-424e-b500-2a84228cd6b2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.405173 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c2462-fefc-4402-9f30-d17b83f17210 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.417524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02049ec3-45e6-437e-921d-305b0bbb1493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.418193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02049ec3-45e6-437e-921d-305b0bbb1493 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.418478 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02049ec3-45e6-437e-921d-305b0bbb1493 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.418755 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02049ec3-45e6-437e-921d-305b0bbb1493 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.419434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02049ec3-45e6-437e-921d-305b0bbb1493 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.420837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02049ec3-45e6-437e-921d-305b0bbb1493 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.434145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 449/895] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:52:01.434145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02049ec3-45e6-437e-921d-305b0bbb1493 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.450242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40726ab-7964-4890-9a8c-b9d72f7e4512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.451611 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40726ab-7964-4890-9a8c-b9d72f7e4512 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.452162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40726ab-7964-4890-9a8c-b9d72f7e4512 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.453057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40726ab-7964-4890-9a8c-b9d72f7e4512 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.454864 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40726ab-7964-4890-9a8c-b9d72f7e4512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:01.455486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40726ab-7964-4890-9a8c-b9d72f7e4512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.457901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40726ab-7964-4890-9a8c-b9d72f7e4512 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.483021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 447/896] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:01.484520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40726ab-7964-4890-9a8c-b9d72f7e4512 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.560662 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02049ec3-45e6-437e-921d-305b0bbb1493 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.602423 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6c59b-99c5-4713-8f6e-e950f6e72cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.603265 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6c59b-99c5-4713-8f6e-e950f6e72cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.603557 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6c59b-99c5-4713-8f6e-e950f6e72cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.603859 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6c59b-99c5-4713-8f6e-e950f6e72cb2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.604562 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6c59b-99c5-4713-8f6e-e950f6e72cb2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.606096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6c59b-99c5-4713-8f6e-e950f6e72cb2 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.613959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40726ab-7964-4890-9a8c-b9d72f7e4512 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.615475 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5ac6c59b-99c5-4713-8f6e-e950f6e72cb2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:01.618910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:01.619932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:01.620863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 450/897] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:01.621659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492c597-fb08-468f-a86f-d3b92baf085a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.621734 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492c597-fb08-468f-a86f-d3b92baf085a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.622256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492c597-fb08-468f-a86f-d3b92baf085a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.623421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 451/898] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:01.627697 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492c597-fb08-468f-a86f-d3b92baf085a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.631714 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:01.659485 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9659c-96d8-4867-bef8-a29c1ca71856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.659485 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9659c-96d8-4867-bef8-a29c1ca71856 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.659485 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9659c-96d8-4867-bef8-a29c1ca71856 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.659485 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9659c-96d8-4867-bef8-a29c1ca71856 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.665502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9659c-96d8-4867-bef8-a29c1ca71856 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.666971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9659c-96d8-4867-bef8-a29c1ca71856 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-40e9659c-96d8-4867-bef8-a29c1ca71856 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:52:01.682129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:01.683283 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:01.684300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 448/899] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:01.684936 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025304e-1eb2-4555-ab9d-5169a1ff8617 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.685006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025304e-1eb2-4555-ab9d-5169a1ff8617 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.685627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025304e-1eb2-4555-ab9d-5169a1ff8617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.702232 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3025304e-1eb2-4555-ab9d-5169a1ff8617 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:01.703412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3025304e-1eb2-4555-ab9d-5169a1ff8617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:01.744715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492c597-fb08-468f-a86f-d3b92baf085a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.776771 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 449/900] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:01.779029 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877585-1c49-4193-afde-cff76f2d1b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.779708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877585-1c49-4193-afde-cff76f2d1b89 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.780001 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877585-1c49-4193-afde-cff76f2d1b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.780314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877585-1c49-4193-afde-cff76f2d1b89 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.781031 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46877585-1c49-4193-afde-cff76f2d1b89 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:01.781344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46877585-1c49-4193-afde-cff76f2d1b89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.783727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46877585-1c49-4193-afde-cff76f2d1b89 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.785665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025304e-1eb2-4555-ab9d-5169a1ff8617 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.799614 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 452/901] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:01.800589 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877585-1c49-4193-afde-cff76f2d1b89 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.871914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:01.888094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:01.897144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025304e-1eb2-4555-ab9d-5169a1ff8617 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.908715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877585-1c49-4193-afde-cff76f2d1b89 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:01.931815 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb367b1-4df7-420f-9c0c-fba24588f029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.932458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb367b1-4df7-420f-9c0c-fba24588f029 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.932797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb367b1-4df7-420f-9c0c-fba24588f029 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.933075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb367b1-4df7-420f-9c0c-fba24588f029 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.933921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb367b1-4df7-420f-9c0c-fba24588f029 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:01.934233 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb367b1-4df7-420f-9c0c-fba24588f029 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.935687 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb367b1-4df7-420f-9c0c-fba24588f029 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.945004 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417c2cf-d1dc-4f77-bba9-700c2b8c6c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:01.945671 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417c2cf-d1dc-4f77-bba9-700c2b8c6c70 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:01.945959 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417c2cf-d1dc-4f77-bba9-700c2b8c6c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:01.946241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417c2cf-d1dc-4f77-bba9-700c2b8c6c70 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:01.946968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417c2cf-d1dc-4f77-bba9-700c2b8c6c70 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:01.947294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417c2cf-d1dc-4f77-bba9-700c2b8c6c70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:01.948694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417c2cf-d1dc-4f77-bba9-700c2b8c6c70 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:01.966534 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 453/902] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:01.966534 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417c2cf-d1dc-4f77-bba9-700c2b8c6c70 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:01.966726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 450/903] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:01.966726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb367b1-4df7-420f-9c0c-fba24588f029 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:02.063756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417c2cf-d1dc-4f77-bba9-700c2b8c6c70 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:02.067606 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb367b1-4df7-420f-9c0c-fba24588f029 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:02.095541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:02.101826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a270368-a17f-4655-9859-450de7eb07ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:02.102533 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a270368-a17f-4655-9859-450de7eb07ea None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:02.102842 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a270368-a17f-4655-9859-450de7eb07ea None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:02.103219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a270368-a17f-4655-9859-450de7eb07ea None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:02.103995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a270368-a17f-4655-9859-450de7eb07ea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:02.105651 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a270368-a17f-4655-9859-450de7eb07ea None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:02.109849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b9022-99e6-4000-9f26-a02264941a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:02.109849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b9022-99e6-4000-9f26-a02264941a90 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:02.109849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b9022-99e6-4000-9f26-a02264941a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:02.109849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b9022-99e6-4000-9f26-a02264941a90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:02.127446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:02.176249 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b9022-99e6-4000-9f26-a02264941a90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:02.177779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b9022-99e6-4000-9f26-a02264941a90 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:02.192559 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 454/904] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:52:01 2022] POST /identity/v3/services => generated 241 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:02.193595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a270368-a17f-4655-9859-450de7eb07ea None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:02.307625 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 451/905] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:01 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:52:02.315600 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b9022-99e6-4000-9f26-a02264941a90 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:02.320692 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a270368-a17f-4655-9859-450de7eb07ea None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:02.364258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ebde2-f7af-46bc-9917-61326384c131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:02.366597 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ebde2-f7af-46bc-9917-61326384c131 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:02.367034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ebde2-f7af-46bc-9917-61326384c131 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:02.367462 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ebde2-f7af-46bc-9917-61326384c131 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:02.368310 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ebde2-f7af-46bc-9917-61326384c131 None admin] RBAC: Adding query filter params (interface=public, service_id=9263077f607449428744f25471b30012, region_id=RegionOne) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:02.368636 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ebde2-f7af-46bc-9917-61326384c131 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:02.370026 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ebde2-f7af-46bc-9917-61326384c131 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:02.377667 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 455/906] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 14:52:02 2022] GET /identity/v3/endpoints?service_id=9263077f607449428744f25471b30012&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:02.383791 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ebde2-f7af-46bc-9917-61326384c131 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:02.386852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:02.442111 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:02.455867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b9022-99e6-4000-9f26-a02264941a90 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:02.493651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d243b5-e5cc-4506-8a0c-89077cf556e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:02.494462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d243b5-e5cc-4506-8a0c-89077cf556e2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:02.494756 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d243b5-e5cc-4506-8a0c-89077cf556e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:02.495043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d243b5-e5cc-4506-8a0c-89077cf556e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:02.497077 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ebde2-f7af-46bc-9917-61326384c131 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:02.506163 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d243b5-e5cc-4506-8a0c-89077cf556e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:02.507920 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d243b5-e5cc-4506-8a0c-89077cf556e2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a7d243b5-e5cc-4506-8a0c-89077cf556e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:02.530798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:02.532859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:02.534083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:02.534083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:02.534083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:02.534083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:02.534083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 452/907] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:02.536150 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef139323-d4cd-4583-9b6d-8d703e2ae8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:02.537021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef139323-d4cd-4583-9b6d-8d703e2ae8db None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:02.537404 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef139323-d4cd-4583-9b6d-8d703e2ae8db None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:02.537756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef139323-d4cd-4583-9b6d-8d703e2ae8db None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:02.538220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d243b5-e5cc-4506-8a0c-89077cf556e2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:02.538664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef139323-d4cd-4583-9b6d-8d703e2ae8db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:02.540118 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef139323-d4cd-4583-9b6d-8d703e2ae8db None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:02.547039 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 456/908] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 14:52:02 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:52:02.659854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d243b5-e5cc-4506-8a0c-89077cf556e2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:02.661553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:02.700969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038ac76-9c9b-45d2-b39b-cd2ee10226b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:02.701669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038ac76-9c9b-45d2-b39b-cd2ee10226b1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:02.701957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038ac76-9c9b-45d2-b39b-cd2ee10226b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:02.702240 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038ac76-9c9b-45d2-b39b-cd2ee10226b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:02.703046 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038ac76-9c9b-45d2-b39b-cd2ee10226b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:02.703363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038ac76-9c9b-45d2-b39b-cd2ee10226b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:02.705401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038ac76-9c9b-45d2-b39b-cd2ee10226b1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:02.720076 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 453/909] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:02.725190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef139323-d4cd-4583-9b6d-8d703e2ae8db None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:02.850016 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef139323-d4cd-4583-9b6d-8d703e2ae8db None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:02.866930 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:02.889490 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb853598-3a29-4fbe-b1ce-bdedb22875ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:02.890308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb853598-3a29-4fbe-b1ce-bdedb22875ae None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:02.890594 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb853598-3a29-4fbe-b1ce-bdedb22875ae None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:02.890863 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb853598-3a29-4fbe-b1ce-bdedb22875ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:02.899227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb853598-3a29-4fbe-b1ce-bdedb22875ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:02.900724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb853598-3a29-4fbe-b1ce-bdedb22875ae None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-fb853598-3a29-4fbe-b1ce-bdedb22875ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:52:02.916549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:02.917843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:02.918966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:52:02.918966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:52:02.918966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:52:02.918966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:52:02.918966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:52:02.918966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:52:02.918966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:02.918966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:02.921549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 457/910] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:02.927496 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038ac76-9c9b-45d2-b39b-cd2ee10226b1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:03.036995 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038ac76-9c9b-45d2-b39b-cd2ee10226b1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:03.074140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56320e28-50aa-426a-bc0d-a7c60ef3de5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:03.074812 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56320e28-50aa-426a-bc0d-a7c60ef3de5a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:03.075102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56320e28-50aa-426a-bc0d-a7c60ef3de5a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:03.075387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56320e28-50aa-426a-bc0d-a7c60ef3de5a None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:03.076190 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56320e28-50aa-426a-bc0d-a7c60ef3de5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:03.076536 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56320e28-50aa-426a-bc0d-a7c60ef3de5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:03.077986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56320e28-50aa-426a-bc0d-a7c60ef3de5a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:03.092031 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 454/911] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:52:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:03.098262 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb853598-3a29-4fbe-b1ce-bdedb22875ae None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:03.215784 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb853598-3a29-4fbe-b1ce-bdedb22875ae None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:03.254935 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c7b18-aa9f-45eb-88fc-e2bc46067735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:03.255605 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c7b18-aa9f-45eb-88fc-e2bc46067735 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:03.255895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c7b18-aa9f-45eb-88fc-e2bc46067735 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:03.256177 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c7b18-aa9f-45eb-88fc-e2bc46067735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:03.257309 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c7b18-aa9f-45eb-88fc-e2bc46067735 None admin] RBAC: Adding query filter params (role.id=e8a21e2686c549e5b16fd9df5f041c72, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=eadd75d096ac49758fdf012c8d8755ad) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:03.257626 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c7b18-aa9f-45eb-88fc-e2bc46067735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:03.259090 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c7b18-aa9f-45eb-88fc-e2bc46067735 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:03.265809 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 458/912] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:03 2022] GET /identity/v3/role_assignments?user.id=eadd75d096ac49758fdf012c8d8755ad&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=e8a21e2686c549e5b16fd9df5f041c72 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:03.741076 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dc3f1-a7f2-42c9-84fe-78550052643f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:03.741647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dc3f1-a7f2-42c9-84fe-78550052643f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:03.741647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dc3f1-a7f2-42c9-84fe-78550052643f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:03.743335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 455/913] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:03.749313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a5c9c-63df-44b4-9b34-50f39d4b2a2d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:03.749861 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a5c9c-63df-44b4-9b34-50f39d4b2a2d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:03.749861 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a5c9c-63df-44b4-9b34-50f39d4b2a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:03.768295 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-452a5c9c-63df-44b4-9b34-50f39d4b2a2d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:03.769432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-452a5c9c-63df-44b4-9b34-50f39d4b2a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:03.802613 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:03.841890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 459/914] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:03.864649 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dc3f1-a7f2-42c9-84fe-78550052643f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:03.891888 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:03.971909 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dc3f1-a7f2-42c9-84fe-78550052643f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:04.010252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87a40a-de45-4a60-9b06-3d35dcf7ad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:04.011516 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87a40a-de45-4a60-9b06-3d35dcf7ad1e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:04.011890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87a40a-de45-4a60-9b06-3d35dcf7ad1e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:04.012175 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87a40a-de45-4a60-9b06-3d35dcf7ad1e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:04.012869 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87a40a-de45-4a60-9b06-3d35dcf7ad1e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:04.014315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87a40a-de45-4a60-9b06-3d35dcf7ad1e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9a87a40a-de45-4a60-9b06-3d35dcf7ad1e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:04.027661 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:04.028689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:04.029587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 456/915] 10.222.0.88 () {56 vars in 1132 bytes} [Thu Jun 16 14:52:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:04.036015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a5c9c-63df-44b4-9b34-50f39d4b2a2d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:04.145016 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a5c9c-63df-44b4-9b34-50f39d4b2a2d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:04.181749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc95e05-6091-4615-a200-b0aba63bb957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:04.182785 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc95e05-6091-4615-a200-b0aba63bb957 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:04.183281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc95e05-6091-4615-a200-b0aba63bb957 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:04.183789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc95e05-6091-4615-a200-b0aba63bb957 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:04.184744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc95e05-6091-4615-a200-b0aba63bb957 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:04.185262 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc95e05-6091-4615-a200-b0aba63bb957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:04.186853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc95e05-6091-4615-a200-b0aba63bb957 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:04.192601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 460/916] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:52:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:04.200774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87a40a-de45-4a60-9b06-3d35dcf7ad1e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:04.287222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:04.305632 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87a40a-de45-4a60-9b06-3d35dcf7ad1e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:04.340466 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054246e-a572-4627-bbd6-302c348fe241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:04.341138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054246e-a572-4627-bbd6-302c348fe241 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:04.341428 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054246e-a572-4627-bbd6-302c348fe241 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:04.341713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054246e-a572-4627-bbd6-302c348fe241 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:04.342457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054246e-a572-4627-bbd6-302c348fe241 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:04.342988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054246e-a572-4627-bbd6-302c348fe241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:04.345249 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054246e-a572-4627-bbd6-302c348fe241 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:04.351374 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 457/917] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:52:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:04.356941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc95e05-6091-4615-a200-b0aba63bb957 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:04.393705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:04.467408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc95e05-6091-4615-a200-b0aba63bb957 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:04.509040 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bdab4-1bdf-43b6-99e1-585fbd90aed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:04.509713 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bdab4-1bdf-43b6-99e1-585fbd90aed7 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:04.513192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bdab4-1bdf-43b6-99e1-585fbd90aed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:04.513192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bdab4-1bdf-43b6-99e1-585fbd90aed7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:04.513192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bdab4-1bdf-43b6-99e1-585fbd90aed7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:04.513192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bdab4-1bdf-43b6-99e1-585fbd90aed7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:04.637550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b165a6b-6827-4686-a1fa-808be0090cf1 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:04.637849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b165a6b-6827-4686-a1fa-808be0090cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:04.638191 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b165a6b-6827-4686-a1fa-808be0090cf1 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:04.640360 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 458/918] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:52:04.646339 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726d229-5a19-4bf1-b939-307d7e27db2c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:04.647671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726d229-5a19-4bf1-b939-307d7e27db2c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:04.647671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726d229-5a19-4bf1-b939-307d7e27db2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:04.649279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 461/919] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 14:52:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:04.666391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e726d229-5a19-4bf1-b939-307d7e27db2c None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:04.667662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e726d229-5a19-4bf1-b939-307d7e27db2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:04.756303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 459/920] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:52:04.784345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bdab4-1bdf-43b6-99e1-585fbd90aed7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:04.884574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66621b-57cc-4ef6-a468-7098c2c07b5a None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:04.884863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66621b-57cc-4ef6-a468-7098c2c07b5a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:04.885153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66621b-57cc-4ef6-a468-7098c2c07b5a None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:04.886777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 460/921] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:52:04.889530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:04.892183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46899c5-d83b-49ab-8854-e2ea61ed3dc4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:04.892634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46899c5-d83b-49ab-8854-e2ea61ed3dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:04.893017 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46899c5-d83b-49ab-8854-e2ea61ed3dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:04.905049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bdab4-1bdf-43b6-99e1-585fbd90aed7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:04.913690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d46899c5-d83b-49ab-8854-e2ea61ed3dc4 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:04.914873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d46899c5-d83b-49ab-8854-e2ea61ed3dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:04.951191 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2e46-6141-430d-865e-ad2337b5ec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:04.952006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2e46-6141-430d-865e-ad2337b5ec6d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:04.952315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2e46-6141-430d-865e-ad2337b5ec6d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:04.952596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2e46-6141-430d-865e-ad2337b5ec6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:04.953258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db2e46-6141-430d-865e-ad2337b5ec6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:04.955210 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db2e46-6141-430d-865e-ad2337b5ec6d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-c6db2e46-6141-430d-865e-ad2337b5ec6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:04.970601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:04.971702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:04.972595 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 462/922] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:52:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:04.979494 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2e46-6141-430d-865e-ad2337b5ec6d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:05.009985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 461/923] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:05.034880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46899c5-d83b-49ab-8854-e2ea61ed3dc4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:05.043473 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:05.106330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2e46-6141-430d-865e-ad2337b5ec6d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:05.149349 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46899c5-d83b-49ab-8854-e2ea61ed3dc4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:05.157583 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b3be1-93f2-49e0-9ec5-daadbb1fb3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:05.158364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3be1-93f2-49e0-9ec5-daadbb1fb3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.158791 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3be1-93f2-49e0-9ec5-daadbb1fb3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.159123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3be1-93f2-49e0-9ec5-daadbb1fb3e7 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.160261 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b3be1-93f2-49e0-9ec5-daadbb1fb3e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:05.160377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b3be1-93f2-49e0-9ec5-daadbb1fb3e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:05.161795 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b3be1-93f2-49e0-9ec5-daadbb1fb3e7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:05.172604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 463/924] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:52:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:05.174243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b3be1-93f2-49e0-9ec5-daadbb1fb3e7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:05.193458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bb28a-246e-49b7-af1e-e5300cbc23cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:05.194947 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bb28a-246e-49b7-af1e-e5300cbc23cb None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.195665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bb28a-246e-49b7-af1e-e5300cbc23cb None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.196172 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1bb28a-246e-49b7-af1e-e5300cbc23cb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.196921 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:05.197247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bb28a-246e-49b7-af1e-e5300cbc23cb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:05.199314 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1bb28a-246e-49b7-af1e-e5300cbc23cb None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-8f1bb28a-246e-49b7-af1e-e5300cbc23cb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:05.215230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:05.216348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:52:05.217215 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:05.219543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 462/925] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:52:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:05.224023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bb28a-246e-49b7-af1e-e5300cbc23cb None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:05.292501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b3be1-93f2-49e0-9ec5-daadbb1fb3e7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:05.333223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f93d42-bfa0-40d7-8ef3-f5636d14a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:05.333955 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f93d42-bfa0-40d7-8ef3-f5636d14a752 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.334243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f93d42-bfa0-40d7-8ef3-f5636d14a752 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.334527 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f93d42-bfa0-40d7-8ef3-f5636d14a752 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.335308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f93d42-bfa0-40d7-8ef3-f5636d14a752 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:05.335599 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f93d42-bfa0-40d7-8ef3-f5636d14a752 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:05.337022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f93d42-bfa0-40d7-8ef3-f5636d14a752 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:05.342486 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 464/926] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:52:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:05.345101 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:05.349859 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f93d42-bfa0-40d7-8ef3-f5636d14a752 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:05.363093 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1bb28a-246e-49b7-af1e-e5300cbc23cb None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:05.411166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b14240-2c64-49a5-9e69-02fb53b7767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:05.411166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b14240-2c64-49a5-9e69-02fb53b7767f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.411166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b14240-2c64-49a5-9e69-02fb53b7767f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.411166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b14240-2c64-49a5-9e69-02fb53b7767f None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.411580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b14240-2c64-49a5-9e69-02fb53b7767f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:05.411580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b14240-2c64-49a5-9e69-02fb53b7767f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:05.411580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b14240-2c64-49a5-9e69-02fb53b7767f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:05.416537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 463/927] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:52:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:05.424002 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b14240-2c64-49a5-9e69-02fb53b7767f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:05.461675 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:05.483224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f93d42-bfa0-40d7-8ef3-f5636d14a752 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:05.533892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a73fae-5ff8-44b8-b194-9f92c3e7d62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:05.534565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a73fae-5ff8-44b8-b194-9f92c3e7d62b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.534853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a73fae-5ff8-44b8-b194-9f92c3e7d62b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.535137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a73fae-5ff8-44b8-b194-9f92c3e7d62b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.535937 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a73fae-5ff8-44b8-b194-9f92c3e7d62b None admin] RBAC: Adding query filter params (interface=public, service_id=63c0a3485f9646818f0162039e291b84, region_id=RegionOne) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:05.536319 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a73fae-5ff8-44b8-b194-9f92c3e7d62b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:05.537740 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a73fae-5ff8-44b8-b194-9f92c3e7d62b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:05.538385 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b14240-2c64-49a5-9e69-02fb53b7767f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:05.548403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 465/928] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 14:52:05 2022] GET /identity/v3/endpoints?service_id=63c0a3485f9646818f0162039e291b84&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:05.548403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc8ec8-ec01-435f-bfbe-062c93eaf3fd None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.548403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc8ec8-ec01-435f-bfbe-062c93eaf3fd None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.548403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc8ec8-ec01-435f-bfbe-062c93eaf3fd None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.548403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 466/929] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:05.553850 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc8ec8-ec01-435f-bfbe-062c93eaf3fd None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:05.579538 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf840897-6b3f-4918-9577-8a85550d333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:05.580267 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf840897-6b3f-4918-9577-8a85550d333a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.580526 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf840897-6b3f-4918-9577-8a85550d333a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.580817 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf840897-6b3f-4918-9577-8a85550d333a None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.581517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf840897-6b3f-4918-9577-8a85550d333a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:05.581828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf840897-6b3f-4918-9577-8a85550d333a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:05.583222 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf840897-6b3f-4918-9577-8a85550d333a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:05.585007 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:05.595089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 464/930] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 14:52:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:05.595089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42229ed6-2bdb-4a1e-9ef4-cfeaaf0f397a None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.595089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42229ed6-2bdb-4a1e-9ef4-cfeaaf0f397a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.595089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42229ed6-2bdb-4a1e-9ef4-cfeaaf0f397a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.614759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-42229ed6-2bdb-4a1e-9ef4-cfeaaf0f397a None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:05.615954 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-42229ed6-2bdb-4a1e-9ef4-cfeaaf0f397a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:05.663687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc8ec8-ec01-435f-bfbe-062c93eaf3fd None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:05.692389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 465/931] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:05.693420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42229ed6-2bdb-4a1e-9ef4-cfeaaf0f397a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:05.709529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1382a-5a07-4c96-8d58-e51148f918c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:05.710512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1382a-5a07-4c96-8d58-e51148f918c9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.711006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1382a-5a07-4c96-8d58-e51148f918c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.711488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1382a-5a07-4c96-8d58-e51148f918c9 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.712438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1382a-5a07-4c96-8d58-e51148f918c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:05.714021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1382a-5a07-4c96-8d58-e51148f918c9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:05.721367 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 467/932] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 14:52:05 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:52:05.722512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1382a-5a07-4c96-8d58-e51148f918c9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:05.734506 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:05.849471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42229ed6-2bdb-4a1e-9ef4-cfeaaf0f397a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:05.855599 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1382a-5a07-4c96-8d58-e51148f918c9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:05.881585 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:05.892842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4144de-0d63-4efa-9c7a-12fe6d9f465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:05.892842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4144de-0d63-4efa-9c7a-12fe6d9f465b None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.892842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4144de-0d63-4efa-9c7a-12fe6d9f465b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.893205 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4144de-0d63-4efa-9c7a-12fe6d9f465b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.894757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4144de-0d63-4efa-9c7a-12fe6d9f465b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:05.895707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4144de-0d63-4efa-9c7a-12fe6d9f465b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:05.916011 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697e16b-2003-4e67-bde6-bba25f5d4947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:05.916736 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697e16b-2003-4e67-bde6-bba25f5d4947 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:05.917032 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697e16b-2003-4e67-bde6-bba25f5d4947 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:05.917306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697e16b-2003-4e67-bde6-bba25f5d4947 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:05.929609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697e16b-2003-4e67-bde6-bba25f5d4947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b697e16b-2003-4e67-bde6-bba25f5d4947 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:05.939661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:05.940915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:05.941911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 468/933] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 14:52:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:05.947239 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697e16b-2003-4e67-bde6-bba25f5d4947 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:06.020350 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:06.053337 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 466/934] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 14:52:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:06.067314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697e16b-2003-4e67-bde6-bba25f5d4947 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:06.114585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65974f-ab44-40e9-bdf0-495812ad1136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:06.115567 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65974f-ab44-40e9-bdf0-495812ad1136 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:06.116009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65974f-ab44-40e9-bdf0-495812ad1136 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:06.116474 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65974f-ab44-40e9-bdf0-495812ad1136 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:06.117469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65974f-ab44-40e9-bdf0-495812ad1136 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:06.117941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65974f-ab44-40e9-bdf0-495812ad1136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:06.119556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65974f-ab44-40e9-bdf0-495812ad1136 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:06.131201 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 469/935] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 14:52:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:06.137392 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4144de-0d63-4efa-9c7a-12fe6d9f465b None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:06.260578 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4144de-0d63-4efa-9c7a-12fe6d9f465b None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:06.297641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e5395-48fd-4fc2-9f9d-44ba306fc069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:06.298482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e5395-48fd-4fc2-9f9d-44ba306fc069 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:06.298767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e5395-48fd-4fc2-9f9d-44ba306fc069 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:06.299047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e5395-48fd-4fc2-9f9d-44ba306fc069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:06.310541 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e5395-48fd-4fc2-9f9d-44ba306fc069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:06.312011 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e5395-48fd-4fc2-9f9d-44ba306fc069 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-083e5395-48fd-4fc2-9f9d-44ba306fc069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:06.331106 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:06.332119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:06.333071 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:06.333071 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:06.333071 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:06.333071 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:06.333071 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 467/936] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:06.339613 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65974f-ab44-40e9-bdf0-495812ad1136 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:06.450831 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65974f-ab44-40e9-bdf0-495812ad1136 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:06.479428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:06.489953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9ed8-54e8-4259-9ecb-8c95e880e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:06.490916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9ed8-54e8-4259-9ecb-8c95e880e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:06.491377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9ed8-54e8-4259-9ecb-8c95e880e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:06.514058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9ed8-54e8-4259-9ecb-8c95e880e7e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:06.514058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd9ed8-54e8-4259-9ecb-8c95e880e7e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:06.514058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd9ed8-54e8-4259-9ecb-8c95e880e7e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:06.514058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd9ed8-54e8-4259-9ecb-8c95e880e7e0 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:06.514058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 470/937] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:06.515501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e5395-48fd-4fc2-9f9d-44ba306fc069 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:06.556685 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:06.628477 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e5395-48fd-4fc2-9f9d-44ba306fc069 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:06.666315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374c07c-df51-48b9-9ef0-6efb33b26339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:06.667453 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374c07c-df51-48b9-9ef0-6efb33b26339 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:06.667938 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374c07c-df51-48b9-9ef0-6efb33b26339 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:06.668425 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374c07c-df51-48b9-9ef0-6efb33b26339 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:06.677539 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374c07c-df51-48b9-9ef0-6efb33b26339 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:06.679256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374c07c-df51-48b9-9ef0-6efb33b26339 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-9374c07c-df51-48b9-9ef0-6efb33b26339 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:52:06.695038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:06.696060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:52:06.697126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:06.699388 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 468/938] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:52:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:06.705470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9ed8-54e8-4259-9ecb-8c95e880e7e0 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:06.815053 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9ed8-54e8-4259-9ecb-8c95e880e7e0 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:06.854312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfa2d4-9ff6-48b8-802e-3326d60cafa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:06.855304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfa2d4-9ff6-48b8-802e-3326d60cafa9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:06.855750 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfa2d4-9ff6-48b8-802e-3326d60cafa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:06.856646 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfa2d4-9ff6-48b8-802e-3326d60cafa9 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:06.857646 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfa2d4-9ff6-48b8-802e-3326d60cafa9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:06.858125 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfa2d4-9ff6-48b8-802e-3326d60cafa9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:06.859932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfa2d4-9ff6-48b8-802e-3326d60cafa9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:06.881114 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 471/939] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 14:52:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:06.887555 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374c07c-df51-48b9-9ef0-6efb33b26339 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:06.953326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a3423-5867-4ce5-9b50-77586fd2417f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:06.954965 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a3423-5867-4ce5-9b50-77586fd2417f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:06.955330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a3423-5867-4ce5-9b50-77586fd2417f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:06.957116 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 472/940] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:06.964324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaf9d1-43df-45ec-a666-49b7e97042bb None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:06.964867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaf9d1-43df-45ec-a666-49b7e97042bb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:06.965358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaf9d1-43df-45ec-a666-49b7e97042bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:06.983315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-deeaf9d1-43df-45ec-a666-49b7e97042bb None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:06.985037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-deeaf9d1-43df-45ec-a666-49b7e97042bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:07.005953 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:07.012528 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374c07c-df51-48b9-9ef0-6efb33b26339 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:07.040565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:07.049764 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9821-d3eb-4ca8-a411-50fabe735678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:07.050797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9821-d3eb-4ca8-a411-50fabe735678 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:07.051025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9821-d3eb-4ca8-a411-50fabe735678 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:07.051369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9821-d3eb-4ca8-a411-50fabe735678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:07.063313 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c9821-d3eb-4ca8-a411-50fabe735678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:07.064979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c9821-d3eb-4ca8-a411-50fabe735678 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:07.074805 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 473/941] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ea5c9821-d3eb-4ca8-a411-50fabe735678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:07.089032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:07.089991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:07.090915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:07.090915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:07.090915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:07.090915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:07.093113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 469/942] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:07.098319 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeaf9d1-43df-45ec-a666-49b7e97042bb None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:07.102173 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9821-d3eb-4ca8-a411-50fabe735678 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:07.215941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9821-d3eb-4ca8-a411-50fabe735678 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:07.233465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeaf9d1-43df-45ec-a666-49b7e97042bb None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:07.259617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0f523-b88d-4ba0-896b-02bb9d8fe9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:07.261355 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0f523-b88d-4ba0-896b-02bb9d8fe9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:07.261653 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0f523-b88d-4ba0-896b-02bb9d8fe9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:07.261936 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0f523-b88d-4ba0-896b-02bb9d8fe9b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:07.262671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0f523-b88d-4ba0-896b-02bb9d8fe9b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:07.262984 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0f523-b88d-4ba0-896b-02bb9d8fe9b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:07.264408 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0f523-b88d-4ba0-896b-02bb9d8fe9b5 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:07.280384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774347d4-bda6-4cc8-94e9-d74024f9dd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:07.284271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 470/943] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:07.285857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774347d4-bda6-4cc8-94e9-d74024f9dd0d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:07.286157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774347d4-bda6-4cc8-94e9-d74024f9dd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:07.286450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774347d4-bda6-4cc8-94e9-d74024f9dd0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:07.286686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0f523-b88d-4ba0-896b-02bb9d8fe9b5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:07.287122 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774347d4-bda6-4cc8-94e9-d74024f9dd0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:07.288541 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774347d4-bda6-4cc8-94e9-d74024f9dd0d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:07.300197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-774347d4-bda6-4cc8-94e9-d74024f9dd0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:07.309410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:07.316321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:07.317208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 474/944] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:07.318013 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774347d4-bda6-4cc8-94e9-d74024f9dd0d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:07.363708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:07.423175 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0f523-b88d-4ba0-896b-02bb9d8fe9b5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:07.430835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774347d4-bda6-4cc8-94e9-d74024f9dd0d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:07.467004 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52569fb4-1b34-4a58-b70a-f40c64bec7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:07.467837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52569fb4-1b34-4a58-b70a-f40c64bec7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:07.468258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52569fb4-1b34-4a58-b70a-f40c64bec7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:07.468653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52569fb4-1b34-4a58-b70a-f40c64bec7a3 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:07.468982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80963da-15ab-4ac9-916f-188b908b4e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:07.469636 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52569fb4-1b34-4a58-b70a-f40c64bec7a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:07.470054 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52569fb4-1b34-4a58-b70a-f40c64bec7a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:07.470369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80963da-15ab-4ac9-916f-188b908b4e17 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:07.470799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80963da-15ab-4ac9-916f-188b908b4e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:07.471221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80963da-15ab-4ac9-916f-188b908b4e17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:07.471843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52569fb4-1b34-4a58-b70a-f40c64bec7a3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:07.479689 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80963da-15ab-4ac9-916f-188b908b4e17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:07.481401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80963da-15ab-4ac9-916f-188b908b4e17 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:07.482272 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 475/945] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:52:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:07.483226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52569fb4-1b34-4a58-b70a-f40c64bec7a3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e80963da-15ab-4ac9-916f-188b908b4e17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:52:07.498397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:07.499486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:07.501158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:52:07.501158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:52:07.501158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:52:07.501158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:52:07.501158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:52:07.501158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:52:07.501158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:07.501158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:07.504312 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 471/946] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:07.506646 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80963da-15ab-4ac9-916f-188b908b4e17 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:07.582505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:07.620200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80963da-15ab-4ac9-916f-188b908b4e17 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:07.627084 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52569fb4-1b34-4a58-b70a-f40c64bec7a3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:07.631078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:07.661197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b528-6f06-4306-8187-953506e16805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:07.661865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b528-6f06-4306-8187-953506e16805 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:07.662157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b528-6f06-4306-8187-953506e16805 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:07.662445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72b528-6f06-4306-8187-953506e16805 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:07.663317 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72b528-6f06-4306-8187-953506e16805 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:07.663637 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72b528-6f06-4306-8187-953506e16805 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:07.665124 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72b528-6f06-4306-8187-953506e16805 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:07.677971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072f7a5-979b-4764-95dc-ce229a406572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:07.679158 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072f7a5-979b-4764-95dc-ce229a406572 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:07.679626 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072f7a5-979b-4764-95dc-ce229a406572 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:07.680085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072f7a5-979b-4764-95dc-ce229a406572 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:07.681019 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072f7a5-979b-4764-95dc-ce229a406572 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:07.681512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072f7a5-979b-4764-95dc-ce229a406572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:07.683191 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072f7a5-979b-4764-95dc-ce229a406572 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:07.685964 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 472/947] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:52:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:07.686933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b528-6f06-4306-8187-953506e16805 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:07.689667 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 476/948] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:52:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:07.808208 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72b528-6f06-4306-8187-953506e16805 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:07.839563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:07.849919 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa15ec-c453-4dc5-88b5-b54d8cee00bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:07.851141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa15ec-c453-4dc5-88b5-b54d8cee00bf None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:07.853019 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa15ec-c453-4dc5-88b5-b54d8cee00bf None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:07.853019 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa15ec-c453-4dc5-88b5-b54d8cee00bf None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/eadd75d096ac49758fdf012c8d8755ad/roles/e8a21e2686c549e5b16fd9df5f041c72` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:07.854662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa15ec-c453-4dc5-88b5-b54d8cee00bf None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=eadd75d096ac49758fdf012c8d8755ad, role_id=e8a21e2686c549e5b16fd9df5f041c72)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:07.856367 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa15ec-c453-4dc5-88b5-b54d8cee00bf None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:07.930762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 473/949] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:52:07 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/eadd75d096ac49758fdf012c8d8755ad/roles/e8a21e2686c549e5b16fd9df5f041c72 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:52:08.002189 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bdf56-119e-4dd5-8fae-e329fbb17173 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:08.002723 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bdf56-119e-4dd5-8fae-e329fbb17173 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:08.002723 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bdf56-119e-4dd5-8fae-e329fbb17173 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:08.004505 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 477/950] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:08.010535 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236cb3d-5957-487f-a7f1-3e3f8690c1a0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:08.010866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236cb3d-5957-487f-a7f1-3e3f8690c1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:08.011109 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236cb3d-5957-487f-a7f1-3e3f8690c1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:08.032828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4236cb3d-5957-487f-a7f1-3e3f8690c1a0 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:08.034055 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4236cb3d-5957-487f-a7f1-3e3f8690c1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:08.117327 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 474/951] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:08.140528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bdf56-119e-4dd5-8fae-e329fbb17173 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:08.253067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bdf56-119e-4dd5-8fae-e329fbb17173 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:08.290012 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a76ae-bbfa-445b-b42d-a383deea1418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:08.290825 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a76ae-bbfa-445b-b42d-a383deea1418 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:08.291117 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a76ae-bbfa-445b-b42d-a383deea1418 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:08.291397 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a76ae-bbfa-445b-b42d-a383deea1418 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:08.292163 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a76ae-bbfa-445b-b42d-a383deea1418 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:08.293566 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a76ae-bbfa-445b-b42d-a383deea1418 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-7c7a76ae-bbfa-445b-b42d-a383deea1418 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:08.306190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:08.307227 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.308245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 478/952] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:52:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:08.313425 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236cb3d-5957-487f-a7f1-3e3f8690c1a0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:08.427656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236cb3d-5957-487f-a7f1-3e3f8690c1a0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:08.439521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b66c4-d157-4e01-8e32-8306f714dc2d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:08.439848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b66c4-d157-4e01-8e32-8306f714dc2d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:08.440078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b66c4-d157-4e01-8e32-8306f714dc2d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:08.441674 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 479/953] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:08.446556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a506a9-fd74-4ec7-87ff-8a7b6438d2d2 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:08.446843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a506a9-fd74-4ec7-87ff-8a7b6438d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:08.447311 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a506a9-fd74-4ec7-87ff-8a7b6438d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:08.450334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:08.464087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc9384-0253-4742-9908-ae93fbb28a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:08.471125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc9384-0253-4742-9908-ae93fbb28a9d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:08.471125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc9384-0253-4742-9908-ae93fbb28a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:08.471125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc9384-0253-4742-9908-ae93fbb28a9d None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:08.471393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-53a506a9-fd74-4ec7-87ff-8a7b6438d2d2 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:08.471472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc9384-0253-4742-9908-ae93fbb28a9d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:08.471472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc9384-0253-4742-9908-ae93fbb28a9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:08.471598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-53a506a9-fd74-4ec7-87ff-8a7b6438d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:08.471683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc9384-0253-4742-9908-ae93fbb28a9d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:08.474242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 475/954] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:52:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:08.479798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc9384-0253-4742-9908-ae93fbb28a9d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:08.504113 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:08.569662 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 480/955] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:08.593480 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a506a9-fd74-4ec7-87ff-8a7b6438d2d2 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:08.614214 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc9384-0253-4742-9908-ae93fbb28a9d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:08.655553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008ec01-eb61-4282-b3fb-446ee465ca63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:08.656793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008ec01-eb61-4282-b3fb-446ee465ca63 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:08.657316 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008ec01-eb61-4282-b3fb-446ee465ca63 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:08.657906 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008ec01-eb61-4282-b3fb-446ee465ca63 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:08.658811 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008ec01-eb61-4282-b3fb-446ee465ca63 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:08.659328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008ec01-eb61-4282-b3fb-446ee465ca63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:08.661030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008ec01-eb61-4282-b3fb-446ee465ca63 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:08.671316 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 476/956] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:52:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:08.673470 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008ec01-eb61-4282-b3fb-446ee465ca63 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:08.722768 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a506a9-fd74-4ec7-87ff-8a7b6438d2d2 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:08.726213 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:08.768668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:08.769403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:08.769694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:08.769974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:08.770983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:08.772446 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:08.773823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: WARNING keystone.server.flask.common [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 14:52:08.787748 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008ec01-eb61-4282-b3fb-446ee465ca63 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:08.825400 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d4b5f-85d8-423a-abef-578cc3c4ad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:08.826412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d4b5f-85d8-423a-abef-578cc3c4ad05 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:08.826920 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d4b5f-85d8-423a-abef-578cc3c4ad05 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:08.827551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d4b5f-85d8-423a-abef-578cc3c4ad05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:08.828706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d4b5f-85d8-423a-abef-578cc3c4ad05 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:08.830338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d4b5f-85d8-423a-abef-578cc3c4ad05 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:08.853253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.sql.core [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 14:52:08.853253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 14:52:08.853253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [parameters: {'id': 'e6411e4160f74af68310664b69560f51', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 14:52:08.853253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80420) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e6411e4160f74af68310664b69560f51.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e6411e4160f74af68310664b69560f51. Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:52:08.861708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result.read() Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 14:52:08.866182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application yield resource Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise self.value Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application session.commit() Jun 16 14:52:08.867244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.session.flush() Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application compat.raise_( Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 14:52:08.868209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 14:52:08.869967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application result.read() Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:52:08.871706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [parameters: {'id': 'e6411e4160f74af68310664b69560f51', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:08.875119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:08.877585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e6411e4160f74af68310664b69560f51. Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 481/957] 10.222.0.88 () {60 vars in 1140 bytes} [Thu Jun 16 14:52:08 2022] POST /identity/v3/projects => generated 262 bytes in 270 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:08.879615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:08.891874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:08.936304 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 477/958] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 14:52:08 2022] POST /identity/v3/endpoints => generated 349 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:08.998491 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84de40a-47b4-4a5e-b28c-cca7b0e5f153 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:09.050691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73175b84-03c8-4fb5-b3ac-8a357636a93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:09.051495 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73175b84-03c8-4fb5-b3ac-8a357636a93a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:09.051787 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73175b84-03c8-4fb5-b3ac-8a357636a93a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:09.052066 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73175b84-03c8-4fb5-b3ac-8a357636a93a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:09.059995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73175b84-03c8-4fb5-b3ac-8a357636a93a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:09.061629 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73175b84-03c8-4fb5-b3ac-8a357636a93a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-73175b84-03c8-4fb5-b3ac-8a357636a93a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:52:09.077526 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:09.079779 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:09.080801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:52:09.080801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:52:09.080801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:52:09.080801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:52:09.080801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:52:09.080801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:52:09.080801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:09.080801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:09.080801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 482/959] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:09.085878 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d4b5f-85d8-423a-abef-578cc3c4ad05 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:09.197467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d4b5f-85d8-423a-abef-578cc3c4ad05 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:09.199053 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:09.234440 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea79774-49ab-48c4-918a-5d015b829d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:09.235785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea79774-49ab-48c4-918a-5d015b829d4f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:09.236076 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea79774-49ab-48c4-918a-5d015b829d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:09.236395 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea79774-49ab-48c4-918a-5d015b829d4f None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:09.237158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea79774-49ab-48c4-918a-5d015b829d4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:09.237471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea79774-49ab-48c4-918a-5d015b829d4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:09.238989 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea79774-49ab-48c4-918a-5d015b829d4f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:09.256654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 478/960] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:52:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:10.010699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c24847-3d77-4b8f-81da-877ddee70bca None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:10.011078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c24847-3d77-4b8f-81da-877ddee70bca None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:10.011258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c24847-3d77-4b8f-81da-877ddee70bca None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:10.012937 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 483/961] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:10.019527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63d097-abe1-47d1-b2e4-f5e3f1fca10c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:10.020008 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63d097-abe1-47d1-b2e4-f5e3f1fca10c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:10.020086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63d097-abe1-47d1-b2e4-f5e3f1fca10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:10.038214 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4a63d097-abe1-47d1-b2e4-f5e3f1fca10c None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:10.039392 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4a63d097-abe1-47d1-b2e4-f5e3f1fca10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:10.121255 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 479/962] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:10.144772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c24847-3d77-4b8f-81da-877ddee70bca None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:10.161137 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690b06-5d3b-4567-9050-192421b2e66b None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:10.161802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690b06-5d3b-4567-9050-192421b2e66b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:10.162313 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690b06-5d3b-4567-9050-192421b2e66b None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:10.164019 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 480/963] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:52:10.168943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553776c0-d60f-4100-bac0-4a678f07d6b8 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:10.169235 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553776c0-d60f-4100-bac0-4a678f07d6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:10.169521 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553776c0-d60f-4100-bac0-4a678f07d6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:10.189663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-553776c0-d60f-4100-bac0-4a678f07d6b8 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:10.190875 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-553776c0-d60f-4100-bac0-4a678f07d6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:10.234280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:10.268082 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 481/964] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:52:10.272902 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c24847-3d77-4b8f-81da-877ddee70bca None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:10.291696 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553776c0-d60f-4100-bac0-4a678f07d6b8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:10.309532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0abd68-a9b5-42f0-9dc6-a145c98525e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:10.310645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0abd68-a9b5-42f0-9dc6-a145c98525e9 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:10.311168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0abd68-a9b5-42f0-9dc6-a145c98525e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:10.311663 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0abd68-a9b5-42f0-9dc6-a145c98525e9 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:10.312663 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0abd68-a9b5-42f0-9dc6-a145c98525e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:10.314691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0abd68-a9b5-42f0-9dc6-a145c98525e9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:10.316498 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:10.392814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 484/965] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:52:10 2022] POST /identity/v3/services => generated 242 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:10.399275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553776c0-d60f-4100-bac0-4a678f07d6b8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:10.453775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd84599-d12d-41be-8b9b-ba668e9e8867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:10.454686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd84599-d12d-41be-8b9b-ba668e9e8867 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:10.454976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd84599-d12d-41be-8b9b-ba668e9e8867 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:10.455258 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd84599-d12d-41be-8b9b-ba668e9e8867 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:10.469795 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd84599-d12d-41be-8b9b-ba668e9e8867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-8fd84599-d12d-41be-8b9b-ba668e9e8867 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:10.478813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:10.479842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:10.480739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 482/966] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 14:52:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:10.486820 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0abd68-a9b5-42f0-9dc6-a145c98525e9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:10.606319 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0abd68-a9b5-42f0-9dc6-a145c98525e9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:10.638973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:10.645478 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21d493-3120-439f-b078-186838f64451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:10.646148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21d493-3120-439f-b078-186838f64451 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:10.646435 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21d493-3120-439f-b078-186838f64451 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:10.646712 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21d493-3120-439f-b078-186838f64451 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:10.647470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21d493-3120-439f-b078-186838f64451 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:10.647780 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21d493-3120-439f-b078-186838f64451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:10.649169 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21d493-3120-439f-b078-186838f64451 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:10.658549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 485/967] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 14:52:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:10.665330 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd84599-d12d-41be-8b9b-ba668e9e8867 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:10.758108 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:10.781103 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd84599-d12d-41be-8b9b-ba668e9e8867 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:10.816995 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c79df3-64f3-4244-af27-51afe557bd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:10.817789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c79df3-64f3-4244-af27-51afe557bd41 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:10.818075 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c79df3-64f3-4244-af27-51afe557bd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:10.818358 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c79df3-64f3-4244-af27-51afe557bd41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:10.830090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c79df3-64f3-4244-af27-51afe557bd41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:10.831566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c79df3-64f3-4244-af27-51afe557bd41 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-11c79df3-64f3-4244-af27-51afe557bd41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:10.849438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:10.850416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:10.851399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:10.851399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:10.851399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:10.851399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:10.851399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 483/968] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:10.857416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21d493-3120-439f-b078-186838f64451 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:10.969082 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21d493-3120-439f-b078-186838f64451 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:11.005800 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e18d82-09b3-4848-aec6-c498bda2092f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:11.006731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e18d82-09b3-4848-aec6-c498bda2092f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.007021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e18d82-09b3-4848-aec6-c498bda2092f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.007304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e18d82-09b3-4848-aec6-c498bda2092f None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.008035 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e18d82-09b3-4848-aec6-c498bda2092f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:11.008387 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e18d82-09b3-4848-aec6-c498bda2092f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:11.009759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e18d82-09b3-4848-aec6-c498bda2092f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:11.023467 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 486/969] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:11.028889 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c79df3-64f3-4244-af27-51afe557bd41 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:11.137392 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c79df3-64f3-4244-af27-51afe557bd41 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:11.169429 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:11.175891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5875e2-6aeb-4a69-a559-cc1c04283fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:11.176739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5875e2-6aeb-4a69-a559-cc1c04283fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.177030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5875e2-6aeb-4a69-a559-cc1c04283fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.177316 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5875e2-6aeb-4a69-a559-cc1c04283fd6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.185446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5875e2-6aeb-4a69-a559-cc1c04283fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:11.188300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5875e2-6aeb-4a69-a559-cc1c04283fd6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ae5875e2-6aeb-4a69-a559-cc1c04283fd6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:52:11.202647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:11.204033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:11.205310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 484/970] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:52:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:11.213336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e18d82-09b3-4848-aec6-c498bda2092f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:11.275506 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:11.327657 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e18d82-09b3-4848-aec6-c498bda2092f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:11.348145 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43ce8-679a-4b06-b788-56b6196d3ba5 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.348474 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43ce8-679a-4b06-b788-56b6196d3ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.348765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a43ce8-679a-4b06-b788-56b6196d3ba5 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.350374 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 485/971] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:11.355206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8dac-23ec-417a-8eb1-1fa120764694 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.355498 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8dac-23ec-417a-8eb1-1fa120764694 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.355782 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8dac-23ec-417a-8eb1-1fa120764694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.366505 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221b1b7-bb4c-4493-a632-3a9fb572f910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:11.367501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221b1b7-bb4c-4493-a632-3a9fb572f910 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.368072 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221b1b7-bb4c-4493-a632-3a9fb572f910 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.368628 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221b1b7-bb4c-4493-a632-3a9fb572f910 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.369674 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221b1b7-bb4c-4493-a632-3a9fb572f910 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:11.370204 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221b1b7-bb4c-4493-a632-3a9fb572f910 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:11.371868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221b1b7-bb4c-4493-a632-3a9fb572f910 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:11.373273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-b1cb8dac-23ec-417a-8eb1-1fa120764694 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:11.374456 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-b1cb8dac-23ec-417a-8eb1-1fa120764694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:11.398056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 487/972] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 14:52:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:11.399655 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221b1b7-bb4c-4493-a632-3a9fb572f910 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:11.452435 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 486/973] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:11.470220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb8dac-23ec-417a-8eb1-1fa120764694 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:11.507100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221b1b7-bb4c-4493-a632-3a9fb572f910 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:11.540725 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:11.550333 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e29ce-266c-4cfe-ac5c-b940a0eb8fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:11.559519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e29ce-266c-4cfe-ac5c-b940a0eb8fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.559519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e29ce-266c-4cfe-ac5c-b940a0eb8fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.559519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e29ce-266c-4cfe-ac5c-b940a0eb8fd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.579793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb8dac-23ec-417a-8eb1-1fa120764694 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:11.602003 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:11.618729 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4e332-817d-4351-9d39-719080c4f969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:11.620129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e332-817d-4351-9d39-719080c4f969 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.620493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e332-817d-4351-9d39-719080c4f969 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.620828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4e332-817d-4351-9d39-719080c4f969 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.621769 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4e332-817d-4351-9d39-719080c4f969 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:11.624503 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4e332-817d-4351-9d39-719080c4f969 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:11.627499 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e29ce-266c-4cfe-ac5c-b940a0eb8fd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:11.629073 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e29ce-266c-4cfe-ac5c-b940a0eb8fd7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f9d4e332-817d-4351-9d39-719080c4f969 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:11.640134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:11.641846 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:11.642901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:11.649089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 487/974] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:52:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:11.649089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208d4f7-d204-4c72-a3f6-2c8818fc1c1d None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.649089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208d4f7-d204-4c72-a3f6-2c8818fc1c1d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.649089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208d4f7-d204-4c72-a3f6-2c8818fc1c1d None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.652504 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 488/975] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:11.653303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208d4f7-d204-4c72-a3f6-2c8818fc1c1d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:11.762597 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 488/976] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:11 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:52:11.764675 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e61db-583b-4383-8f81-152a199551b8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.764975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e61db-583b-4383-8f81-152a199551b8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.765319 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e61db-583b-4383-8f81-152a199551b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.768187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208d4f7-d204-4c72-a3f6-2c8818fc1c1d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:11.786335 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-444e61db-583b-4383-8f81-152a199551b8 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:11.787981 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-444e61db-583b-4383-8f81-152a199551b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:11.795994 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:11.806849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66078fa0-2eda-4298-a58e-0f17ae4bb023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:11.810420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078fa0-2eda-4298-a58e-0f17ae4bb023 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.810420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078fa0-2eda-4298-a58e-0f17ae4bb023 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.810420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078fa0-2eda-4298-a58e-0f17ae4bb023 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.810420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66078fa0-2eda-4298-a58e-0f17ae4bb023 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:11.810420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66078fa0-2eda-4298-a58e-0f17ae4bb023 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:11.810851 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66078fa0-2eda-4298-a58e-0f17ae4bb023 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:11.820604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 489/977] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:52:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:11.820604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66078fa0-2eda-4298-a58e-0f17ae4bb023 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:11.867501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 489/978] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:11.869062 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e61db-583b-4383-8f81-152a199551b8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:11.887018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:11.946115 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66078fa0-2eda-4298-a58e-0f17ae4bb023 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:11.973069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e61db-583b-4383-8f81-152a199551b8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:11.983784 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02373099-17ce-47f3-8699-8d7cf864e011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:11.984774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02373099-17ce-47f3-8699-8d7cf864e011 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:11.985193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02373099-17ce-47f3-8699-8d7cf864e011 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:11.985575 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02373099-17ce-47f3-8699-8d7cf864e011 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:11.996536 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02373099-17ce-47f3-8699-8d7cf864e011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:11.997996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02373099-17ce-47f3-8699-8d7cf864e011 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.012660 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc40297-926c-49f1-b745-45853b75786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.012885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc40297-926c-49f1-b745-45853b75786b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.013216 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc40297-926c-49f1-b745-45853b75786b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.013597 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc40297-926c-49f1-b745-45853b75786b None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.014413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc40297-926c-49f1-b745-45853b75786b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:12.014848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc40297-926c-49f1-b745-45853b75786b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-02373099-17ce-47f3-8699-8d7cf864e011 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:12.015257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:12.016974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:12.018328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:12.018328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:12.018328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:12.018328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:12.018328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 490/979] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:12.018612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02373099-17ce-47f3-8699-8d7cf864e011 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:12.022704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc40297-926c-49f1-b745-45853b75786b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.028170 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 490/980] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:52:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:12.029002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc40297-926c-49f1-b745-45853b75786b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:12.039908 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:12.135404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:12.139842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02373099-17ce-47f3-8699-8d7cf864e011 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:12.142535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc40297-926c-49f1-b745-45853b75786b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:12.177523 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.178453 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.178898 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.179189 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.179890 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.180064 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938310d-35d1-422f-9e0d-9495a0be00b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.180761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938310d-35d1-422f-9e0d-9495a0be00b1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.181053 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938310d-35d1-422f-9e0d-9495a0be00b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.181337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938310d-35d1-422f-9e0d-9495a0be00b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.181445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.182064 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938310d-35d1-422f-9e0d-9495a0be00b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:12.182378 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938310d-35d1-422f-9e0d-9495a0be00b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.182666 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: WARNING keystone.server.flask.common [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 14:52:12.183753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938310d-35d1-422f-9e0d-9495a0be00b1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.198917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 491/981] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:12.199687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938310d-35d1-422f-9e0d-9495a0be00b1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:12.230460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.sql.core [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 14:52:12.230460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 14:52:12.230460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [parameters: {'user_id': '2234d32569b843779faf1989c2209335', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 14:52:12.230460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80419) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application result.read() Jun 16 14:52:12.247004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:52:12.247979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application yield resource Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise self.value Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application session.commit() Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 14:52:12.249004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.session.flush() Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application compat.raise_( Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 14:52:12.251174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:52:12.252373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application result.read() Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:52:12.253265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [parameters: {'user_id': '2234d32569b843779faf1989c2209335', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:12.257018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 14:52:12.258527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 14:52:12.259744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 14:52:12.259744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 14:52:12.259744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 14:52:12.259744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 14:52:12.259744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 14:52:12.259744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:12.259744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 491/982] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 14:52:12 2022] POST /identity/v3/users => generated 160 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:12.259744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:12.281747 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:12.317092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938310d-35d1-422f-9e0d-9495a0be00b1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:12.358283 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3dec1-4e02-4387-9d4d-18fcd461dfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.358972 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3dec1-4e02-4387-9d4d-18fcd461dfbb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.359829 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3dec1-4e02-4387-9d4d-18fcd461dfbb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.360109 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3dec1-4e02-4387-9d4d-18fcd461dfbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.360948 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3dec1-4e02-4387-9d4d-18fcd461dfbb None admin] RBAC: Adding query filter params (interface=admin, service_id=63c0a3485f9646818f0162039e291b84, region_id=RegionOne) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:12.361269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3dec1-4e02-4387-9d4d-18fcd461dfbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.362784 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3dec1-4e02-4387-9d4d-18fcd461dfbb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.364478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7665e-d9ca-41b0-a3a5-f82b4f363863 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:12.373007 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 492/983] 10.222.0.88 () {56 vars in 1249 bytes} [Thu Jun 16 14:52:12 2022] GET /identity/v3/endpoints?service_id=63c0a3485f9646818f0162039e291b84&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:12.373007 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3dec1-4e02-4387-9d4d-18fcd461dfbb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:12.404293 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7848c9-2a52-4999-8f4d-04ca8cf06361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.405094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7848c9-2a52-4999-8f4d-04ca8cf06361 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.405386 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7848c9-2a52-4999-8f4d-04ca8cf06361 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.405770 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7848c9-2a52-4999-8f4d-04ca8cf06361 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.412871 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7848c9-2a52-4999-8f4d-04ca8cf06361 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.414566 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7848c9-2a52-4999-8f4d-04ca8cf06361 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ba7848c9-2a52-4999-8f4d-04ca8cf06361 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:52:12.429318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:12.430506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:12.431934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:52:12.431934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:52:12.431934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:52:12.431934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:52:12.431934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:52:12.431934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:52:12.431934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:12.431934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:12.435294 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 492/984] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:12.436024 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7848c9-2a52-4999-8f4d-04ca8cf06361 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:12.462946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:12.481997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3dec1-4e02-4387-9d4d-18fcd461dfbb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:12.519936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:12.526281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d0a68-4b1f-4f44-946e-104b1ad6c6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.527189 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d0a68-4b1f-4f44-946e-104b1ad6c6ba None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.527546 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d0a68-4b1f-4f44-946e-104b1ad6c6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.527886 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d0a68-4b1f-4f44-946e-104b1ad6c6ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.536131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d0a68-4b1f-4f44-946e-104b1ad6c6ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.537634 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d0a68-4b1f-4f44-946e-104b1ad6c6ba None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-eb5d0a68-4b1f-4f44-946e-104b1ad6c6ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:52:12.553224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:12.554393 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:12.555509 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 493/985] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:52:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:12.558659 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7848c9-2a52-4999-8f4d-04ca8cf06361 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:12.559995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d0a68-4b1f-4f44-946e-104b1ad6c6ba None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:12.595926 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aecd16-3d59-428d-a70d-5ee8a7e8fab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.596731 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aecd16-3d59-428d-a70d-5ee8a7e8fab7 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.597023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aecd16-3d59-428d-a70d-5ee8a7e8fab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.597318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aecd16-3d59-428d-a70d-5ee8a7e8fab7 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.597990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aecd16-3d59-428d-a70d-5ee8a7e8fab7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.599379 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aecd16-3d59-428d-a70d-5ee8a7e8fab7 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.616265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 493/986] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 14:52:12 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:52:12.616265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aecd16-3d59-428d-a70d-5ee8a7e8fab7 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:12.691554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d0a68-4b1f-4f44-946e-104b1ad6c6ba None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:12.731906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c60af-5884-414e-9915-ecc83ae5028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.732652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c60af-5884-414e-9915-ecc83ae5028e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.733022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c60af-5884-414e-9915-ecc83ae5028e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.733385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c60af-5884-414e-9915-ecc83ae5028e None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.734786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c60af-5884-414e-9915-ecc83ae5028e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:12.735180 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c60af-5884-414e-9915-ecc83ae5028e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.736768 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c60af-5884-414e-9915-ecc83ae5028e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.742007 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:12.750847 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 494/987] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:52:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:12.755178 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e853f-2708-47c7-bf18-8a2fe3ee1acf None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.755178 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e853f-2708-47c7-bf18-8a2fe3ee1acf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.755178 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e853f-2708-47c7-bf18-8a2fe3ee1acf None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.755178 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 495/988] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:12.757050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e853f-2708-47c7-bf18-8a2fe3ee1acf None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:12.766731 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aecd16-3d59-428d-a70d-5ee8a7e8fab7 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:12.810745 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5dda5-3a77-4cc8-9505-a2719f5293b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.811576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5dda5-3a77-4cc8-9505-a2719f5293b6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.811865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5dda5-3a77-4cc8-9505-a2719f5293b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.812147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5dda5-3a77-4cc8-9505-a2719f5293b6 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.813745 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5dda5-3a77-4cc8-9505-a2719f5293b6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:12.814128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5dda5-3a77-4cc8-9505-a2719f5293b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.814256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:12.815501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5dda5-3a77-4cc8-9505-a2719f5293b6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.852061 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 494/989] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 14:52:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:12.853543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85942d5e-32ab-4380-bd9c-6d68c7fd3e77 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.853836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85942d5e-32ab-4380-bd9c-6d68c7fd3e77 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.854120 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85942d5e-32ab-4380-bd9c-6d68c7fd3e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.874280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e853f-2708-47c7-bf18-8a2fe3ee1acf None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:12.877675 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-85942d5e-32ab-4380-bd9c-6d68c7fd3e77 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:12.880551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-85942d5e-32ab-4380-bd9c-6d68c7fd3e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:12.930448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344f0f-499b-4184-8397-0066faf53e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:12.932086 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344f0f-499b-4184-8397-0066faf53e2c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:12.932414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344f0f-499b-4184-8397-0066faf53e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:12.932699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344f0f-499b-4184-8397-0066faf53e2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:12.933558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7344f0f-499b-4184-8397-0066faf53e2c None admin] RBAC: Adding query filter params (role.id=e8a21e2686c549e5b16fd9df5f041c72, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=eadd75d096ac49758fdf012c8d8755ad) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:12.933878 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7344f0f-499b-4184-8397-0066faf53e2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:12.935358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7344f0f-499b-4184-8397-0066faf53e2c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:12.944530 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 496/990] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:12 2022] GET /identity/v3/role_assignments?user.id=eadd75d096ac49758fdf012c8d8755ad&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=e8a21e2686c549e5b16fd9df5f041c72 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:12.976659 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 495/991] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:13.000386 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344f0f-499b-4184-8397-0066faf53e2c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:13.125900 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344f0f-499b-4184-8397-0066faf53e2c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:13.140778 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:13.165348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb979479-8136-41c7-875f-16f59b3e48ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:13.166164 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb979479-8136-41c7-875f-16f59b3e48ab None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:13.166455 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb979479-8136-41c7-875f-16f59b3e48ab None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:13.166739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb979479-8136-41c7-875f-16f59b3e48ab None admin] PATH_INFO: `/v3/services/compute` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:13.167408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb979479-8136-41c7-875f-16f59b3e48ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:13.168835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb979479-8136-41c7-875f-16f59b3e48ab None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-eb979479-8136-41c7-875f-16f59b3e48ab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:13.182123 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:13.183226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:13.184573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 497/992] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:13.189406 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85942d5e-32ab-4380-bd9c-6d68c7fd3e77 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:13.267611 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:13.293899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85942d5e-32ab-4380-bd9c-6d68c7fd3e77 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:13.330652 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea0583-cc33-4e64-8faa-4ba28c8c1f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:13.331422 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea0583-cc33-4e64-8faa-4ba28c8c1f20 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:13.331785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea0583-cc33-4e64-8faa-4ba28c8c1f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:13.332140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea0583-cc33-4e64-8faa-4ba28c8c1f20 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:13.332949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea0583-cc33-4e64-8faa-4ba28c8c1f20 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:13.333337 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea0583-cc33-4e64-8faa-4ba28c8c1f20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:13.334787 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea0583-cc33-4e64-8faa-4ba28c8c1f20 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:13.344788 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 496/993] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:52:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:13.347368 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb979479-8136-41c7-875f-16f59b3e48ab None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:13.457970 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb979479-8136-41c7-875f-16f59b3e48ab None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:13.497285 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b84034-6faf-4d7c-9194-ac3d94e88b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:13.497957 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b84034-6faf-4d7c-9194-ac3d94e88b0b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:13.498240 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b84034-6faf-4d7c-9194-ac3d94e88b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:13.498519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b84034-6faf-4d7c-9194-ac3d94e88b0b None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:13.499232 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b84034-6faf-4d7c-9194-ac3d94e88b0b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:13.499538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b84034-6faf-4d7c-9194-ac3d94e88b0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:13.502093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b84034-6faf-4d7c-9194-ac3d94e88b0b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:13.507688 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 498/994] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:52:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:13.513423 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea0583-cc33-4e64-8faa-4ba28c8c1f20 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:13.621362 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea0583-cc33-4e64-8faa-4ba28c8c1f20 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:13.674822 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a8e5c-8f58-4d34-b0d6-820d6396f34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:13.675484 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8e5c-8f58-4d34-b0d6-820d6396f34a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:13.675768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8e5c-8f58-4d34-b0d6-820d6396f34a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:13.676044 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8e5c-8f58-4d34-b0d6-820d6396f34a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:13.676865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a8e5c-8f58-4d34-b0d6-820d6396f34a None admin] RBAC: Adding query filter params (interface=public, service_id=ff609282b7f8487d9a8203e471dcefd9, region_id=RegionOne) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:13.677178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a8e5c-8f58-4d34-b0d6-820d6396f34a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:13.678548 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a8e5c-8f58-4d34-b0d6-820d6396f34a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:13.688665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 497/995] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 14:52:13 2022] GET /identity/v3/endpoints?service_id=ff609282b7f8487d9a8203e471dcefd9&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:13.689851 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b84034-6faf-4d7c-9194-ac3d94e88b0b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:13.734990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:13.798045 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b84034-6faf-4d7c-9194-ac3d94e88b0b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:13.834759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e14c6-1fb3-416e-ab6d-12b0378c1a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:13.835420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e14c6-1fb3-416e-ab6d-12b0378c1a0c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:13.835702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e14c6-1fb3-416e-ab6d-12b0378c1a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:13.835980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e14c6-1fb3-416e-ab6d-12b0378c1a0c None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:13.836691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e14c6-1fb3-416e-ab6d-12b0378c1a0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:13.838045 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e14c6-1fb3-416e-ab6d-12b0378c1a0c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:13.845279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 499/996] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 14:52:13 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:52:14.832039 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba427f-a372-4aef-9553-7b8631507cc2 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:14.832739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba427f-a372-4aef-9553-7b8631507cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:14.832739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba427f-a372-4aef-9553-7b8631507cc2 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:14.834434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 498/997] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:14.839660 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b029951-ae73-4c54-ba53-3e06c60c2ba6 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:14.840068 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b029951-ae73-4c54-ba53-3e06c60c2ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:14.840250 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b029951-ae73-4c54-ba53-3e06c60c2ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:14.857835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3b029951-ae73-4c54-ba53-3e06c60c2ba6 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:14.859071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3b029951-ae73-4c54-ba53-3e06c60c2ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:14.930495 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 500/998] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:14.954424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba427f-a372-4aef-9553-7b8631507cc2 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:14.957385 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:15.059217 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba427f-a372-4aef-9553-7b8631507cc2 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:15.096812 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f91ca5-4293-428d-a9f3-e54e2a12b8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:15.098028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f91ca5-4293-428d-a9f3-e54e2a12b8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:15.098534 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f91ca5-4293-428d-a9f3-e54e2a12b8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:15.099036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f91ca5-4293-428d-a9f3-e54e2a12b8d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:15.099921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f91ca5-4293-428d-a9f3-e54e2a12b8d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:15.101620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f91ca5-4293-428d-a9f3-e54e2a12b8d0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-06f91ca5-4293-428d-a9f3-e54e2a12b8d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:15.115892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:15.117243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:52:15.118325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:15.121304 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 499/999] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 14:52:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:15.127979 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b029951-ae73-4c54-ba53-3e06c60c2ba6 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:15.162329 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45a1ce-b511-4108-9ba2-65854c637dec None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:15.162654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45a1ce-b511-4108-9ba2-65854c637dec None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:15.162942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45a1ce-b511-4108-9ba2-65854c637dec None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:15.164643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 500/1000] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:52:15.166344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:15.169266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fbeb1-4322-46b0-a923-8e15a9251daa None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:15.169560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fbeb1-4322-46b0-a923-8e15a9251daa None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:15.169841 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fbeb1-4322-46b0-a923-8e15a9251daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:15.186883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-464fbeb1-4322-46b0-a923-8e15a9251daa None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:15.188100 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-464fbeb1-4322-46b0-a923-8e15a9251daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:15.237745 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b029951-ae73-4c54-ba53-3e06c60c2ba6 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:15.254446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:15.271796 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697b142-a8b9-428c-a3b3-52c4d7150388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:15.272520 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697b142-a8b9-428c-a3b3-52c4d7150388 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:15.272706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 501/1001] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:52:15.274687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697b142-a8b9-428c-a3b3-52c4d7150388 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:15.274687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697b142-a8b9-428c-a3b3-52c4d7150388 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:15.274687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697b142-a8b9-428c-a3b3-52c4d7150388 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:15.274687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697b142-a8b9-428c-a3b3-52c4d7150388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:15.275438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697b142-a8b9-428c-a3b3-52c4d7150388 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:15.285764 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 501/1002] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:52:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:15.285914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fbeb1-4322-46b0-a923-8e15a9251daa None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:15.286564 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697b142-a8b9-428c-a3b3-52c4d7150388 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:15.387316 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fbeb1-4322-46b0-a923-8e15a9251daa None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:15.387928 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:15.392045 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697b142-a8b9-428c-a3b3-52c4d7150388 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:15.423259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfe666-5164-4617-b7d3-f52bcdf0022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:15.424036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfe666-5164-4617-b7d3-f52bcdf0022e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:15.424350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfe666-5164-4617-b7d3-f52bcdf0022e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:15.424629 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfe666-5164-4617-b7d3-f52bcdf0022e None admin] PATH_INFO: `/v3/services/image` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:15.425292 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfe666-5164-4617-b7d3-f52bcdf0022e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:15.426654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfe666-5164-4617-b7d3-f52bcdf0022e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:15.429776 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae4dcf-9ae1-4a78-aeda-edea4e4d45d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:15.430608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae4dcf-9ae1-4a78-aeda-edea4e4d45d7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:15.431043 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae4dcf-9ae1-4a78-aeda-edea4e4d45d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:15.431488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae4dcf-9ae1-4a78-aeda-edea4e4d45d7 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:15.432426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae4dcf-9ae1-4a78-aeda-edea4e4d45d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:15.432990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae4dcf-9ae1-4a78-aeda-edea4e4d45d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:15.434548 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae4dcf-9ae1-4a78-aeda-edea4e4d45d7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-13bfe666-5164-4617-b7d3-f52bcdf0022e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:15.438918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:15.440144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:15.441078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 502/1003] 10.222.0.88 () {56 vars in 1105 bytes} [Thu Jun 16 14:52:15 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:15.442010 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 502/1004] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 14:52:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:15.445962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfe666-5164-4617-b7d3-f52bcdf0022e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:15.449840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae4dcf-9ae1-4a78-aeda-edea4e4d45d7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:15.511284 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:15.567303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfe666-5164-4617-b7d3-f52bcdf0022e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:15.569918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae4dcf-9ae1-4a78-aeda-edea4e4d45d7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:15.603239 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994197b-c70e-431e-9a10-653c12fc307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:15.603906 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994197b-c70e-431e-9a10-653c12fc307f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:15.604187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994197b-c70e-431e-9a10-653c12fc307f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:15.604493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994197b-c70e-431e-9a10-653c12fc307f None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:15.605199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994197b-c70e-431e-9a10-653c12fc307f None admin] RBAC: Adding query filter params (name=image) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:15.605504 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994197b-c70e-431e-9a10-653c12fc307f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:15.606339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dae200-bae0-4fc9-be5a-8830a249ee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:15.606874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994197b-c70e-431e-9a10-653c12fc307f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:15.607137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dae200-bae0-4fc9-be5a-8830a249ee0e None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:15.607497 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dae200-bae0-4fc9-be5a-8830a249ee0e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:15.607859 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dae200-bae0-4fc9-be5a-8830a249ee0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:15.608654 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dae200-bae0-4fc9-be5a-8830a249ee0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:15.610532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dae200-bae0-4fc9-be5a-8830a249ee0e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:15.613062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 503/1005] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:15.618289 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994197b-c70e-431e-9a10-653c12fc307f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:15.690056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 503/1006] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 14:52:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:52:15.733197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994197b-c70e-431e-9a10-653c12fc307f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:15.760265 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:15.780732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8aa47-909a-43a3-9a72-a198122d7374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:15.782328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8aa47-909a-43a3-9a72-a198122d7374 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:15.782328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8aa47-909a-43a3-9a72-a198122d7374 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:15.782618 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8aa47-909a-43a3-9a72-a198122d7374 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:15.783312 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8aa47-909a-43a3-9a72-a198122d7374 None admin] RBAC: Adding query filter params (type=image) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:15.783627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8aa47-909a-43a3-9a72-a198122d7374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:15.785056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8aa47-909a-43a3-9a72-a198122d7374 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:15.790537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 504/1007] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:16.017655 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6b7c7-5e4d-4216-9576-4a6e5bbea19f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:16.018148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6b7c7-5e4d-4216-9576-4a6e5bbea19f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:16.018148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6b7c7-5e4d-4216-9576-4a6e5bbea19f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:16.019529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 504/1008] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:16.024819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba54660-9e38-49c1-a2ab-7352cb2a7a1d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:16.025253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba54660-9e38-49c1-a2ab-7352cb2a7a1d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:16.025431 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba54660-9e38-49c1-a2ab-7352cb2a7a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:16.042948 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3ba54660-9e38-49c1-a2ab-7352cb2a7a1d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:16.044135 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3ba54660-9e38-49c1-a2ab-7352cb2a7a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:16.113192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 505/1009] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:16 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:16.136897 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6b7c7-5e4d-4216-9576-4a6e5bbea19f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:16.163556 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:16.246217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6b7c7-5e4d-4216-9576-4a6e5bbea19f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:16.283333 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12690135-a1eb-4228-b643-64800398cb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:16.284208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12690135-a1eb-4228-b643-64800398cb27 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:16.284552 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12690135-a1eb-4228-b643-64800398cb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:16.285079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12690135-a1eb-4228-b643-64800398cb27 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:16.285741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12690135-a1eb-4228-b643-64800398cb27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:16.287162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12690135-a1eb-4228-b643-64800398cb27 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-12690135-a1eb-4228-b643-64800398cb27 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:16.299263 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:16.300840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:16.301726 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 505/1010] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:16.306610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba54660-9e38-49c1-a2ab-7352cb2a7a1d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:16.409203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba54660-9e38-49c1-a2ab-7352cb2a7a1d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:16.423748 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:16.444389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9846e8-3191-410f-aeb6-d273f7dddce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:16.445060 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9846e8-3191-410f-aeb6-d273f7dddce2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:16.445346 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9846e8-3191-410f-aeb6-d273f7dddce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:16.445626 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9846e8-3191-410f-aeb6-d273f7dddce2 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:16.446336 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9846e8-3191-410f-aeb6-d273f7dddce2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:16.446647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9846e8-3191-410f-aeb6-d273f7dddce2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:16.448032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9846e8-3191-410f-aeb6-d273f7dddce2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:16.458494 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 506/1011] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:52:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:16.459641 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12690135-a1eb-4228-b643-64800398cb27 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:16.562999 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:16.582027 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12690135-a1eb-4228-b643-64800398cb27 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:16.616853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38678e21-2a5c-44f0-8754-26bd490d9bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:16.617518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38678e21-2a5c-44f0-8754-26bd490d9bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:16.617830 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38678e21-2a5c-44f0-8754-26bd490d9bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:16.618081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38678e21-2a5c-44f0-8754-26bd490d9bc5 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:16.618788 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38678e21-2a5c-44f0-8754-26bd490d9bc5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:16.619096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38678e21-2a5c-44f0-8754-26bd490d9bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:16.620486 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38678e21-2a5c-44f0-8754-26bd490d9bc5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:16.630553 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 506/1012] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 14:52:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:16.632077 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9846e8-3191-410f-aeb6-d273f7dddce2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:16.731547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9846e8-3191-410f-aeb6-d273f7dddce2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:16.767518 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c70385-7bf0-40f9-a5ed-47f7b679e722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:16.768489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c70385-7bf0-40f9-a5ed-47f7b679e722 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:16.768987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c70385-7bf0-40f9-a5ed-47f7b679e722 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:16.769459 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c70385-7bf0-40f9-a5ed-47f7b679e722 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:16.770369 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c70385-7bf0-40f9-a5ed-47f7b679e722 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:16.771915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c70385-7bf0-40f9-a5ed-47f7b679e722 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:16.854664 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 507/1013] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 14:52:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:17.873459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291df32e-6cda-4f26-9672-a044131a7da9 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:17.873989 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291df32e-6cda-4f26-9672-a044131a7da9 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:17.873989 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291df32e-6cda-4f26-9672-a044131a7da9 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:17.875528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 507/1014] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:17.880911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9dd2-3145-44a0-b938-4fcee8328566 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:17.881617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9dd2-3145-44a0-b938-4fcee8328566 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:17.881788 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9dd2-3145-44a0-b938-4fcee8328566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:17.901056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2f2e9dd2-3145-44a0-b938-4fcee8328566 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:17.902417 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2f2e9dd2-3145-44a0-b938-4fcee8328566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:17.934645 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:17.969711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fc696-dea3-4ba1-bd6a-d36f1ca219f3 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:17.970031 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fc696-dea3-4ba1-bd6a-d36f1ca219f3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:17.970271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fc696-dea3-4ba1-bd6a-d36f1ca219f3 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:17.971794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 508/1015] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:52:17.978045 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af9a1d-2ebb-4d61-b0d6-0ac38f1178a4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:17.978386 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af9a1d-2ebb-4d61-b0d6-0ac38f1178a4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:17.978681 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af9a1d-2ebb-4d61-b0d6-0ac38f1178a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:17.986217 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 508/1016] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:17 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:17.998694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a1af9a1d-2ebb-4d61-b0d6-0ac38f1178a4 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:18.001384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a1af9a1d-2ebb-4d61-b0d6-0ac38f1178a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:18.010536 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9dd2-3145-44a0-b938-4fcee8328566 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:18.078537 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 509/1017] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:17 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:52:18.100933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af9a1d-2ebb-4d61-b0d6-0ac38f1178a4 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:18.127569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9dd2-3145-44a0-b938-4fcee8328566 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:18.166020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3055c1-1e81-4084-ba0b-5586a947e585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:18.167127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3055c1-1e81-4084-ba0b-5586a947e585 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:18.167547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3055c1-1e81-4084-ba0b-5586a947e585 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:18.167831 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3055c1-1e81-4084-ba0b-5586a947e585 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:18.169033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3055c1-1e81-4084-ba0b-5586a947e585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:18.170476 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3055c1-1e81-4084-ba0b-5586a947e585 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:18.173099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 509/1018] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:18.178380 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3055c1-1e81-4084-ba0b-5586a947e585 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:18.196525 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:18.207829 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:18.217277 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af9a1d-2ebb-4d61-b0d6-0ac38f1178a4 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:18.258516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6270a-c069-4642-9107-badba03ca352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:18.259982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6270a-c069-4642-9107-badba03ca352 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:18.260546 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6270a-c069-4642-9107-badba03ca352 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:18.261050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6270a-c069-4642-9107-badba03ca352 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:18.262340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6270a-c069-4642-9107-badba03ca352 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:18.264612 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6270a-c069-4642-9107-badba03ca352 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:18.286500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3055c1-1e81-4084-ba0b-5586a947e585 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:18.320310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c243c3b-eba9-4bb9-b4d5-536100304d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:18.320968 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c243c3b-eba9-4bb9-b4d5-536100304d3d None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:18.321356 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c243c3b-eba9-4bb9-b4d5-536100304d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:18.321640 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c243c3b-eba9-4bb9-b4d5-536100304d3d None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:18.322350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c243c3b-eba9-4bb9-b4d5-536100304d3d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:18.323778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c243c3b-eba9-4bb9-b4d5-536100304d3d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:18.373579 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 510/1019] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:52:18 2022] POST /identity/v3/services => generated 242 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:52:18.475565 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 510/1020] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:52:18 2022] POST /identity/v3/projects => generated 311 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:52:19.133421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08593a-2d7b-4b8f-a44d-d2f4465ca291 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:19.133996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08593a-2d7b-4b8f-a44d-d2f4465ca291 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:19.133996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08593a-2d7b-4b8f-a44d-d2f4465ca291 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:19.135484 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 511/1021] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:19.140754 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ee8a7-8214-4e22-9258-ec666471f47d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:19.141197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ee8a7-8214-4e22-9258-ec666471f47d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:19.141375 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ee8a7-8214-4e22-9258-ec666471f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:19.161196 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c42ee8a7-8214-4e22-9258-ec666471f47d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:19.162387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c42ee8a7-8214-4e22-9258-ec666471f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:19.240486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 511/1022] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:19.263656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08593a-2d7b-4b8f-a44d-d2f4465ca291 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:19.367794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08593a-2d7b-4b8f-a44d-d2f4465ca291 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:19.402364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a8773-7bfb-4f3b-9ec3-89c32411367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:19.403111 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a8773-7bfb-4f3b-9ec3-89c32411367b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:19.403400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a8773-7bfb-4f3b-9ec3-89c32411367b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:19.403682 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a8773-7bfb-4f3b-9ec3-89c32411367b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:19.414127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a8773-7bfb-4f3b-9ec3-89c32411367b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-473a8773-7bfb-4f3b-9ec3-89c32411367b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:19.423034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:19.423998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:19.425284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:19.426347 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 512/1023] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 14:52:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:19.430363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ee8a7-8214-4e22-9258-ec666471f47d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:19.438704 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:19.550560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ee8a7-8214-4e22-9258-ec666471f47d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:19.588300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae85e9-f8ac-4301-83cd-a0ddaba47826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:19.589338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae85e9-f8ac-4301-83cd-a0ddaba47826 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:19.589860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae85e9-f8ac-4301-83cd-a0ddaba47826 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:19.590371 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae85e9-f8ac-4301-83cd-a0ddaba47826 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:19.591343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae85e9-f8ac-4301-83cd-a0ddaba47826 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:19.591859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae85e9-f8ac-4301-83cd-a0ddaba47826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:19.593767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae85e9-f8ac-4301-83cd-a0ddaba47826 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:19.603797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 512/1024] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 14:52:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:19.609855 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a8773-7bfb-4f3b-9ec3-89c32411367b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:19.612869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:19.722565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a8773-7bfb-4f3b-9ec3-89c32411367b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:19.759316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51682390-6757-4df7-bec3-3335d349f448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:19.760119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51682390-6757-4df7-bec3-3335d349f448 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:19.760429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51682390-6757-4df7-bec3-3335d349f448 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:19.760747 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51682390-6757-4df7-bec3-3335d349f448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:19.772125 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51682390-6757-4df7-bec3-3335d349f448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:19.773713 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51682390-6757-4df7-bec3-3335d349f448 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-51682390-6757-4df7-bec3-3335d349f448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:19.791739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:19.792926 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:19.793853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:19.793853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:19.793853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:19.793853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:19.796567 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 513/1025] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:19.798883 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae85e9-f8ac-4301-83cd-a0ddaba47826 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:19.869316 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:19.905068 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae85e9-f8ac-4301-83cd-a0ddaba47826 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:19.940164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af7efd-e5e2-4127-9298-5eae9d693121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:19.940876 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7efd-e5e2-4127-9298-5eae9d693121 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:19.941163 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7efd-e5e2-4127-9298-5eae9d693121 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:19.941441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7efd-e5e2-4127-9298-5eae9d693121 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:19.942173 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af7efd-e5e2-4127-9298-5eae9d693121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:19.942482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af7efd-e5e2-4127-9298-5eae9d693121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:19.943853 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af7efd-e5e2-4127-9298-5eae9d693121 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:19.961098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 513/1026] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:19.962346 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51682390-6757-4df7-bec3-3335d349f448 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:20.021601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:20.068930 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51682390-6757-4df7-bec3-3335d349f448 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:20.103305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d28b81-c0cc-4d29-9687-af795e8e59da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:20.104106 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d28b81-c0cc-4d29-9687-af795e8e59da None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:20.104425 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d28b81-c0cc-4d29-9687-af795e8e59da None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:20.104709 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d28b81-c0cc-4d29-9687-af795e8e59da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:20.113154 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d28b81-c0cc-4d29-9687-af795e8e59da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:20.114600 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d28b81-c0cc-4d29-9687-af795e8e59da None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-76d28b81-c0cc-4d29-9687-af795e8e59da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:52:20.130006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:20.131318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:52:20.132354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:20.137781 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 514/1027] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 14:52:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:20.138901 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af7efd-e5e2-4127-9298-5eae9d693121 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:20.247733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af7efd-e5e2-4127-9298-5eae9d693121 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:20.276487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:20.285230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78113412-74e9-429e-bdb7-db850198e302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:20.285871 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78113412-74e9-429e-bdb7-db850198e302 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:20.291193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78113412-74e9-429e-bdb7-db850198e302 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:20.291193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78113412-74e9-429e-bdb7-db850198e302 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:20.291193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78113412-74e9-429e-bdb7-db850198e302 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:20.291193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78113412-74e9-429e-bdb7-db850198e302 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:20.291193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78113412-74e9-429e-bdb7-db850198e302 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:20.313798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 514/1028] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 14:52:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:20.315987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d28b81-c0cc-4d29-9687-af795e8e59da None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:20.422071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d28b81-c0cc-4d29-9687-af795e8e59da None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:20.436166 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:20.458193 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7155b-ebad-4f29-8a21-4348bf4cacce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:20.458856 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7155b-ebad-4f29-8a21-4348bf4cacce None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:20.459147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7155b-ebad-4f29-8a21-4348bf4cacce None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:20.459430 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7155b-ebad-4f29-8a21-4348bf4cacce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:20.547339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7155b-ebad-4f29-8a21-4348bf4cacce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:20.548570 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7155b-ebad-4f29-8a21-4348bf4cacce None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:20.573294 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0442d9a-fe12-4537-9a70-a53f2cb9f6ee None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:20.573591 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0442d9a-fe12-4537-9a70-a53f2cb9f6ee None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:20.573876 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0442d9a-fe12-4537-9a70-a53f2cb9f6ee None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:20.575387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 515/1029] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:20.581885 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b023-a896-49eb-88dd-a7bbee94d3bf None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:20.582180 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b023-a896-49eb-88dd-a7bbee94d3bf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:20.582468 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b023-a896-49eb-88dd-a7bbee94d3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:20.600632 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0859b023-a896-49eb-88dd-a7bbee94d3bf None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:20.601839 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0859b023-a896-49eb-88dd-a7bbee94d3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:20.672353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 515/1030] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:20 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:52:20.677587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 516/1031] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:20.678064 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7155b-ebad-4f29-8a21-4348bf4cacce None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:20.696055 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:20.703743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b023-a896-49eb-88dd-a7bbee94d3bf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:20.793384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7155b-ebad-4f29-8a21-4348bf4cacce None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:20.808323 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b023-a896-49eb-88dd-a7bbee94d3bf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:20.824974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:20.829946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d3774-ea6b-4ac7-8285-69e6df4e0a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:20.830742 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d3774-ea6b-4ac7-8285-69e6df4e0a1d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:20.831033 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d3774-ea6b-4ac7-8285-69e6df4e0a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:20.831313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d3774-ea6b-4ac7-8285-69e6df4e0a1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:20.844778 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d3774-ea6b-4ac7-8285-69e6df4e0a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:20.846139 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2795e12-96c2-47a5-8395-5a46769991c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:20.846139 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2795e12-96c2-47a5-8395-5a46769991c0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:20.846139 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2795e12-96c2-47a5-8395-5a46769991c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:20.846139 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2795e12-96c2-47a5-8395-5a46769991c0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:20.846529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d3774-ea6b-4ac7-8285-69e6df4e0a1d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:20.846819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2795e12-96c2-47a5-8395-5a46769991c0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:20.848454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2795e12-96c2-47a5-8395-5a46769991c0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e2795e12-96c2-47a5-8395-5a46769991c0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:20.863750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:20.869929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:20.870907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 517/1032] 10.222.0.88 () {56 vars in 1105 bytes} [Thu Jun 16 14:52:20 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:20.871736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f2882-81f4-43bd-82e1-177587422699 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:20.871736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f2882-81f4-43bd-82e1-177587422699 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:20.871736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f2882-81f4-43bd-82e1-177587422699 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:20.871736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 518/1033] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-bb7d3774-ea6b-4ac7-8285-69e6df4e0a1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:20.871988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:20.877436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:20.878420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:20.878420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:20.878420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:20.878420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:20.878647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f2882-81f4-43bd-82e1-177587422699 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:20.878732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 516/1034] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:20.878732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713ed2e-fafa-4ad1-a7e8-bcdc3fbbd6ae None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:20.878732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713ed2e-fafa-4ad1-a7e8-bcdc3fbbd6ae None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:20.878732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713ed2e-fafa-4ad1-a7e8-bcdc3fbbd6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:20.894730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-d713ed2e-fafa-4ad1-a7e8-bcdc3fbbd6ae None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:20.895946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-d713ed2e-fafa-4ad1-a7e8-bcdc3fbbd6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:20.965950 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:20.980457 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 517/1035] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:20.981397 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713ed2e-fafa-4ad1-a7e8-bcdc3fbbd6ae None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:21.001256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f2882-81f4-43bd-82e1-177587422699 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.042765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2220e-5cec-43d0-969f-1100053a55b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.043438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2220e-5cec-43d0-969f-1100053a55b0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.043728 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2220e-5cec-43d0-969f-1100053a55b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.044012 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2220e-5cec-43d0-969f-1100053a55b0 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.044768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2220e-5cec-43d0-969f-1100053a55b0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:21.045078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2220e-5cec-43d0-969f-1100053a55b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.047036 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2220e-5cec-43d0-969f-1100053a55b0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.052274 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 519/1036] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:21.053092 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2220e-5cec-43d0-969f-1100053a55b0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:21.105500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:21.108078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713ed2e-fafa-4ad1-a7e8-bcdc3fbbd6ae None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.144387 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ad5d6-39b8-4e90-bcbb-969e09c5c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.145069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ad5d6-39b8-4e90-bcbb-969e09c5c55f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.145314 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ad5d6-39b8-4e90-bcbb-969e09c5c55f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.145588 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ad5d6-39b8-4e90-bcbb-969e09c5c55f None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.146308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ad5d6-39b8-4e90-bcbb-969e09c5c55f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:21.146618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ad5d6-39b8-4e90-bcbb-969e09c5c55f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.147996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ad5d6-39b8-4e90-bcbb-969e09c5c55f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.160759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 518/1037] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:21.161436 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ad5d6-39b8-4e90-bcbb-969e09c5c55f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:21.165411 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2220e-5cec-43d0-969f-1100053a55b0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.200643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3f454-5889-47ba-a691-257e8b1393d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.201436 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3f454-5889-47ba-a691-257e8b1393d2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.201726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3f454-5889-47ba-a691-257e8b1393d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.202005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3f454-5889-47ba-a691-257e8b1393d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.203363 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3f454-5889-47ba-a691-257e8b1393d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.204853 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3f454-5889-47ba-a691-257e8b1393d2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.206497 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 520/1038] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:21.207262 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3f454-5889-47ba-a691-257e8b1393d2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:21.227633 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:21.273637 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ad5d6-39b8-4e90-bcbb-969e09c5c55f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.308176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760d2ee-10aa-452d-a3d0-2c5f1e36b8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.308872 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760d2ee-10aa-452d-a3d0-2c5f1e36b8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.309087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3f454-5889-47ba-a691-257e8b1393d2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.309544 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760d2ee-10aa-452d-a3d0-2c5f1e36b8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.309650 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760d2ee-10aa-452d-a3d0-2c5f1e36b8c7 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.310339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760d2ee-10aa-452d-a3d0-2c5f1e36b8c7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:21.310651 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760d2ee-10aa-452d-a3d0-2c5f1e36b8c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.312678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760d2ee-10aa-452d-a3d0-2c5f1e36b8c7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.317715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 519/1039] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:21.318480 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760d2ee-10aa-452d-a3d0-2c5f1e36b8c7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:21.329590 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:21.351704 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc48ef-7bd0-4bfd-a982-0d20de2c5a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.352553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc48ef-7bd0-4bfd-a982-0d20de2c5a33 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.352840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc48ef-7bd0-4bfd-a982-0d20de2c5a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.353126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc48ef-7bd0-4bfd-a982-0d20de2c5a33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.360866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc48ef-7bd0-4bfd-a982-0d20de2c5a33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.362666 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc48ef-7bd0-4bfd-a982-0d20de2c5a33 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-77fc48ef-7bd0-4bfd-a982-0d20de2c5a33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:52:21.379533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:21.380542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:21.381808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:52:21.381808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:52:21.381808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:52:21.381808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:52:21.381808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:52:21.381808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:52:21.381808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:21.381808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:21.384383 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 521/1040] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:21.385193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc48ef-7bd0-4bfd-a982-0d20de2c5a33 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:21.424512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760d2ee-10aa-452d-a3d0-2c5f1e36b8c7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.445301 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:21.459907 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741fc53-a988-41a4-a8a5-7df679daaadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.460991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741fc53-a988-41a4-a8a5-7df679daaadc None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.461394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741fc53-a988-41a4-a8a5-7df679daaadc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.461813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741fc53-a988-41a4-a8a5-7df679daaadc None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.463048 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741fc53-a988-41a4-a8a5-7df679daaadc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.464851 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741fc53-a988-41a4-a8a5-7df679daaadc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.487597 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc48ef-7bd0-4bfd-a982-0d20de2c5a33 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.522548 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26f917-1cb5-41be-a171-c2013c2c0dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.523212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26f917-1cb5-41be-a171-c2013c2c0dba None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.523498 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26f917-1cb5-41be-a171-c2013c2c0dba None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.523778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26f917-1cb5-41be-a171-c2013c2c0dba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.524606 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26f917-1cb5-41be-a171-c2013c2c0dba None admin] RBAC: Adding query filter params (interface=public, service_id=569aa12d32f24b85a65407353d4cbe8c, region_id=RegionOne) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:21.524921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26f917-1cb5-41be-a171-c2013c2c0dba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.526302 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26f917-1cb5-41be-a171-c2013c2c0dba None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.531594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 522/1041] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 14:52:21 2022] GET /identity/v3/endpoints?service_id=569aa12d32f24b85a65407353d4cbe8c&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:21.532403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26f917-1cb5-41be-a171-c2013c2c0dba None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:21.560353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 520/1042] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:52:21 2022] POST /identity/v3/users => generated 285 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:21.561406 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741fc53-a988-41a4-a8a5-7df679daaadc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:21.569667 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:21.649828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26f917-1cb5-41be-a171-c2013c2c0dba None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.681069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741fc53-a988-41a4-a8a5-7df679daaadc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.695084 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199c0b9-5a4c-4f3e-9423-6363e2c79783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.695768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199c0b9-5a4c-4f3e-9423-6363e2c79783 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.696186 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199c0b9-5a4c-4f3e-9423-6363e2c79783 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.696479 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199c0b9-5a4c-4f3e-9423-6363e2c79783 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.697400 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199c0b9-5a4c-4f3e-9423-6363e2c79783 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:21.697713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199c0b9-5a4c-4f3e-9423-6363e2c79783 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.699191 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199c0b9-5a4c-4f3e-9423-6363e2c79783 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.714739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 523/1043] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:21.721292 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1b2ea-fac8-4001-b666-9d2a0a46c6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.721292 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1b2ea-fac8-4001-b666-9d2a0a46c6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.721292 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1b2ea-fac8-4001-b666-9d2a0a46c6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.721292 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1b2ea-fac8-4001-b666-9d2a0a46c6d4 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.721596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1b2ea-fac8-4001-b666-9d2a0a46c6d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.721989 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199c0b9-5a4c-4f3e-9423-6363e2c79783 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:21.722833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1b2ea-fac8-4001-b666-9d2a0a46c6d4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.730476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 521/1044] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 14:52:21 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:52:21.847291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:21.860268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199c0b9-5a4c-4f3e-9423-6363e2c79783 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:21.893517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67f428-7a5d-424f-9ae9-1a55394ff5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:21.894392 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67f428-7a5d-424f-9ae9-1a55394ff5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:21.895003 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67f428-7a5d-424f-9ae9-1a55394ff5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:21.895286 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67f428-7a5d-424f-9ae9-1a55394ff5d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:21.896126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67f428-7a5d-424f-9ae9-1a55394ff5d3 None admin] RBAC: Adding query filter params (role.id=e8a21e2686c549e5b16fd9df5f041c72, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=5368284e42d84e77a6fdb88c2e8dcd3d) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:21.896470 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67f428-7a5d-424f-9ae9-1a55394ff5d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:21.897912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67f428-7a5d-424f-9ae9-1a55394ff5d3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:21.904159 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 524/1045] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:21 2022] GET /identity/v3/role_assignments?user.id=5368284e42d84e77a6fdb88c2e8dcd3d&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=e8a21e2686c549e5b16fd9df5f041c72 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:23.832921 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dddd1d-0647-4a4e-bb4d-c87ad812e818 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:23.833481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dddd1d-0647-4a4e-bb4d-c87ad812e818 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:23.833481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dddd1d-0647-4a4e-bb4d-c87ad812e818 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:23.835000 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 522/1046] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:23.840680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d0b7e-f6ca-441e-a13a-e44a10821c14 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:23.841153 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d0b7e-f6ca-441e-a13a-e44a10821c14 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:23.841261 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d0b7e-f6ca-441e-a13a-e44a10821c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:23.859152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c27d0b7e-f6ca-441e-a13a-e44a10821c14 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:23.860431 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c27d0b7e-f6ca-441e-a13a-e44a10821c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:23.938976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3349e8-9e57-4b8a-9688-b68c69dfe81e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:23.939267 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3349e8-9e57-4b8a-9688-b68c69dfe81e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:23.939767 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3349e8-9e57-4b8a-9688-b68c69dfe81e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:23.941296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 525/1047] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:23.941593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 523/1048] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:23.946891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdea03-91b7-48d3-b823-8775d19fd4d7 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:23.947189 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdea03-91b7-48d3-b823-8775d19fd4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:23.947477 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdea03-91b7-48d3-b823-8775d19fd4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:23.964807 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1dbdea03-91b7-48d3-b823-8775d19fd4d7 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:23.965762 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3349e8-9e57-4b8a-9688-b68c69dfe81e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:23.965992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1dbdea03-91b7-48d3-b823-8775d19fd4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:24.015313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:24.072447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 526/1049] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:24.083458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3349e8-9e57-4b8a-9688-b68c69dfe81e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.090236 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7e57c-5d88-40fc-9c87-f5d4be34e4c5 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.090532 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7e57c-5d88-40fc-9c87-f5d4be34e4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.091302 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7e57c-5d88-40fc-9c87-f5d4be34e4c5 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.093430 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 527/1050] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:52:24.099064 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7e57c-5d88-40fc-9c87-f5d4be34e4c5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.119594 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4733dc-a492-4e19-99e0-f4ddbb900518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.120442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4733dc-a492-4e19-99e0-f4ddbb900518 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.120742 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4733dc-a492-4e19-99e0-f4ddbb900518 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.120873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:24.121021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4733dc-a492-4e19-99e0-f4ddbb900518 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.132534 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4733dc-a492-4e19-99e0-f4ddbb900518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2d4733dc-a492-4e19-99e0-f4ddbb900518 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:24.140821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:24.141837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:24.142694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 524/1051] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:52:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:24.143422 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cdf3c-11a3-45f3-b169-049433cd0aad None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.143697 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cdf3c-11a3-45f3-b169-049433cd0aad None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.143961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cdf3c-11a3-45f3-b169-049433cd0aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.160497 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ce6cdf3c-11a3-45f3-b169-049433cd0aad None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:24.161717 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ce6cdf3c-11a3-45f3-b169-049433cd0aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:24.210869 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7e57c-5d88-40fc-9c87-f5d4be34e4c5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.235036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 525/1052] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:24.235887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cdf3c-11a3-45f3-b169-049433cd0aad None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.253767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c7a06-d837-4a8a-ab64-a75a0b2c0d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.254570 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7a06-d837-4a8a-ab64-a75a0b2c0d88 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.254858 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7a06-d837-4a8a-ab64-a75a0b2c0d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.255143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7a06-d837-4a8a-ab64-a75a0b2c0d88 None admin] PATH_INFO: `/v3/services/image` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.256056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c7a06-d837-4a8a-ab64-a75a0b2c0d88 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:24.257533 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c7a06-d837-4a8a-ab64-a75a0b2c0d88 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:24.268148 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3c6c7a06-d837-4a8a-ab64-a75a0b2c0d88 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:24.269962 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:24.270996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:24.271859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 528/1053] 10.222.0.88 () {56 vars in 1105 bytes} [Thu Jun 16 14:52:24 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:24.272702 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c7a06-d837-4a8a-ab64-a75a0b2c0d88 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.343620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cdf3c-11a3-45f3-b169-049433cd0aad None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.363367 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:24.378882 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814c7f5-2692-4150-ab3a-0d25c9eb3d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.379563 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814c7f5-2692-4150-ab3a-0d25c9eb3d0b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.379852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814c7f5-2692-4150-ab3a-0d25c9eb3d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.381247 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814c7f5-2692-4150-ab3a-0d25c9eb3d0b None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.381352 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c7a06-d837-4a8a-ab64-a75a0b2c0d88 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.381459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814c7f5-2692-4150-ab3a-0d25c9eb3d0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:24.381647 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814c7f5-2692-4150-ab3a-0d25c9eb3d0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:24.383040 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814c7f5-2692-4150-ab3a-0d25c9eb3d0b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:24.392110 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 526/1054] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:24.393015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814c7f5-2692-4150-ab3a-0d25c9eb3d0b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.416750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e9656-7c70-4bfc-8a4c-73f599492520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.417571 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e9656-7c70-4bfc-8a4c-73f599492520 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.417865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e9656-7c70-4bfc-8a4c-73f599492520 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.418152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e9656-7c70-4bfc-8a4c-73f599492520 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.428401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e9656-7c70-4bfc-8a4c-73f599492520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-473e9656-7c70-4bfc-8a4c-73f599492520 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:24.437041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:24.438556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 529/1055] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 14:52:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:24.439427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e9656-7c70-4bfc-8a4c-73f599492520 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.488445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:24.497665 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814c7f5-2692-4150-ab3a-0d25c9eb3d0b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.535173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb096d2f-b41c-4b49-80db-2bf0577294c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.535846 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb096d2f-b41c-4b49-80db-2bf0577294c4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.536174 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb096d2f-b41c-4b49-80db-2bf0577294c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.536498 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb096d2f-b41c-4b49-80db-2bf0577294c4 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.537212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb096d2f-b41c-4b49-80db-2bf0577294c4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:24.537527 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb096d2f-b41c-4b49-80db-2bf0577294c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:24.538935 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb096d2f-b41c-4b49-80db-2bf0577294c4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:24.544069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 527/1056] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:24.544814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb096d2f-b41c-4b49-80db-2bf0577294c4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.546602 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e9656-7c70-4bfc-8a4c-73f599492520 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.581386 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042728ae-3540-4706-8d49-4875815063d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.582253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042728ae-3540-4706-8d49-4875815063d1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.582537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042728ae-3540-4706-8d49-4875815063d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.582812 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042728ae-3540-4706-8d49-4875815063d1 None admin] PATH_INFO: `/v3/users/b75c329a379c460d8df77a4158d1c83a` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.596786 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042728ae-3540-4706-8d49-4875815063d1 None admin] RBAC: Authorizing `identity:get_user(user_id=b75c329a379c460d8df77a4158d1c83a)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:24.598159 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042728ae-3540-4706-8d49-4875815063d1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:24.600101 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 530/1057] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:24 2022] GET /identity/v3/users/b75c329a379c460d8df77a4158d1c83a => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:24.600775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042728ae-3540-4706-8d49-4875815063d1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.603220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:24.652056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb096d2f-b41c-4b49-80db-2bf0577294c4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.689083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ef7f2-a7e5-4aaa-88c5-6abd8b6a8a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.689813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ef7f2-a7e5-4aaa-88c5-6abd8b6a8a1d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.690101 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ef7f2-a7e5-4aaa-88c5-6abd8b6a8a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.690383 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ef7f2-a7e5-4aaa-88c5-6abd8b6a8a1d None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.691138 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ef7f2-a7e5-4aaa-88c5-6abd8b6a8a1d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:24.691449 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ef7f2-a7e5-4aaa-88c5-6abd8b6a8a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:24.692855 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ef7f2-a7e5-4aaa-88c5-6abd8b6a8a1d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:24.700907 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042728ae-3540-4706-8d49-4875815063d1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.701432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 528/1058] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 14:52:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:24.702209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ef7f2-a7e5-4aaa-88c5-6abd8b6a8a1d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.722989 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:24.738269 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d85872-8666-4e43-9f2c-cf0481b91a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.738938 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d85872-8666-4e43-9f2c-cf0481b91a08 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.739230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d85872-8666-4e43-9f2c-cf0481b91a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.739521 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d85872-8666-4e43-9f2c-cf0481b91a08 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.740246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d85872-8666-4e43-9f2c-cf0481b91a08 None admin] RBAC: Adding query filter params (type=image) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:24.740585 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d85872-8666-4e43-9f2c-cf0481b91a08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:24.741965 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d85872-8666-4e43-9f2c-cf0481b91a08 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:24.746859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 531/1059] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:24.747726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d85872-8666-4e43-9f2c-cf0481b91a08 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.807661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ef7f2-a7e5-4aaa-88c5-6abd8b6a8a1d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.815553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:24.845037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84477f30-da74-4a36-9e37-6bcd1e70f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.845732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84477f30-da74-4a36-9e37-6bcd1e70f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.846024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84477f30-da74-4a36-9e37-6bcd1e70f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.846308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84477f30-da74-4a36-9e37-6bcd1e70f1a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.849775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d85872-8666-4e43-9f2c-cf0481b91a08 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:24.885665 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb3585-1306-484c-ac8c-4cbed6c8c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:24.886463 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb3585-1306-484c-ac8c-4cbed6c8c39a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:24.886750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb3585-1306-484c-ac8c-4cbed6c8c39a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:24.887027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb3585-1306-484c-ac8c-4cbed6c8c39a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:24.897528 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb3585-1306-484c-ac8c-4cbed6c8c39a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:24.898987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb3585-1306-484c-ac8c-4cbed6c8c39a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:24.915601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84477f30-da74-4a36-9e37-6bcd1e70f1a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-4ddb3585-1306-484c-ac8c-4cbed6c8c39a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:24.916849 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:24.917802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:24.918729 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:24.918729 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:24.918729 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:24.918729 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:24.918986 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84477f30-da74-4a36-9e37-6bcd1e70f1a8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:24.919059 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 532/1060] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:24.919138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb3585-1306-484c-ac8c-4cbed6c8c39a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:24.937398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:25.105982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 529/1061] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:52:25.106732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84477f30-da74-4a36-9e37-6bcd1e70f1a8 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:25.138989 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb3585-1306-484c-ac8c-4cbed6c8c39a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:25.156943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:25.173684 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faee300-18f9-45dc-8b32-9a7286852a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:25.174343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faee300-18f9-45dc-8b32-9a7286852a4f None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:25.174633 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faee300-18f9-45dc-8b32-9a7286852a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:25.174913 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faee300-18f9-45dc-8b32-9a7286852a4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:25.175644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faee300-18f9-45dc-8b32-9a7286852a4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:25.177116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faee300-18f9-45dc-8b32-9a7286852a4f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:25.209203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84477f30-da74-4a36-9e37-6bcd1e70f1a8 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:25.226459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:25.243417 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4e1b0-8704-4b20-b80b-28ddf90c9282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:25.244073 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4e1b0-8704-4b20-b80b-28ddf90c9282 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:25.244390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4e1b0-8704-4b20-b80b-28ddf90c9282 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:25.244667 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4e1b0-8704-4b20-b80b-28ddf90c9282 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:25.245379 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4e1b0-8704-4b20-b80b-28ddf90c9282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:25.245685 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4e1b0-8704-4b20-b80b-28ddf90c9282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:25.247045 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4e1b0-8704-4b20-b80b-28ddf90c9282 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:25.259400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 530/1062] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:25.260177 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4e1b0-8704-4b20-b80b-28ddf90c9282 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:25.269418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 533/1063] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 14:52:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:25.270439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faee300-18f9-45dc-8b32-9a7286852a4f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:25.380493 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4e1b0-8704-4b20-b80b-28ddf90c9282 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:25.391773 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faee300-18f9-45dc-8b32-9a7286852a4f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:25.431364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2f748-3d43-4ef5-9575-18a2383af4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:25.432271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2f748-3d43-4ef5-9575-18a2383af4ca None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:25.432427 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695b647-bfd7-43c0-853f-9b2923364484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:25.432643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2f748-3d43-4ef5-9575-18a2383af4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:25.432850 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2f748-3d43-4ef5-9575-18a2383af4ca None admin] PATH_INFO: `/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:25.433209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695b647-bfd7-43c0-853f-9b2923364484 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:25.433500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695b647-bfd7-43c0-853f-9b2923364484 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:25.434133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695b647-bfd7-43c0-853f-9b2923364484 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:25.434234 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2f748-3d43-4ef5-9575-18a2383af4ca None admin] RBAC: Authorizing `identity:get_project(project_id=2a747f9853b646cab08ed44ee8ecbb1e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:25.435517 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2f748-3d43-4ef5-9575-18a2383af4ca None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:25.438042 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 531/1064] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:52:25 2022] GET /identity/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:25.444377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2f748-3d43-4ef5-9575-18a2383af4ca None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:25.447935 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695b647-bfd7-43c0-853f-9b2923364484 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:25.449129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695b647-bfd7-43c0-853f-9b2923364484 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c695b647-bfd7-43c0-853f-9b2923364484 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:52:25.465021 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:25.466221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:25.467391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 534/1065] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 14:52:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:25.473209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695b647-bfd7-43c0-853f-9b2923364484 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:25.501654 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:25.511750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:25.563102 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2f748-3d43-4ef5-9575-18a2383af4ca None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:25.584970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695b647-bfd7-43c0-853f-9b2923364484 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:25.600661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690963f-ba9f-4daf-b31e-53f5365154fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:25.601323 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690963f-ba9f-4daf-b31e-53f5365154fb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:25.601664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690963f-ba9f-4daf-b31e-53f5365154fb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:25.601883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690963f-ba9f-4daf-b31e-53f5365154fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:25.602718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690963f-ba9f-4daf-b31e-53f5365154fb None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=2a747f9853b646cab08ed44ee8ecbb1e, user.id=b75c329a379c460d8df77a4158d1c83a) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:25.603085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690963f-ba9f-4daf-b31e-53f5365154fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:25.604619 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690963f-ba9f-4daf-b31e-53f5365154fb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:25.620670 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 532/1066] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:25 2022] GET /identity/v3/role_assignments?user.id=b75c329a379c460d8df77a4158d1c83a&scope.project.id=2a747f9853b646cab08ed44ee8ecbb1e&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:25.621176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7679a-47dc-4e93-8a3e-c4a6b71c6cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:25.622023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7679a-47dc-4e93-8a3e-c4a6b71c6cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:25.622422 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7679a-47dc-4e93-8a3e-c4a6b71c6cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:25.622770 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7679a-47dc-4e93-8a3e-c4a6b71c6cd4 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:25.623726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7679a-47dc-4e93-8a3e-c4a6b71c6cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:25.624170 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7679a-47dc-4e93-8a3e-c4a6b71c6cd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:25.626500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7679a-47dc-4e93-8a3e-c4a6b71c6cd4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:25.652568 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 535/1067] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 14:52:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:25.653953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690963f-ba9f-4daf-b31e-53f5365154fb None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:25.769400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690963f-ba9f-4daf-b31e-53f5365154fb None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:25.781413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:25.809952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39eafa-c1ee-44c0-9030-86a14dc95916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:25.811002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39eafa-c1ee-44c0-9030-86a14dc95916 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:25.811522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39eafa-c1ee-44c0-9030-86a14dc95916 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:25.811954 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39eafa-c1ee-44c0-9030-86a14dc95916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:25.824137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39eafa-c1ee-44c0-9030-86a14dc95916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:25.825889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39eafa-c1ee-44c0-9030-86a14dc95916 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1d39eafa-c1ee-44c0-9030-86a14dc95916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:25.844775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:25.846293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:25.847255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:25.847255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:25.847255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:25.847255 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:25.847493 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 533/1068] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:25.852754 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7679a-47dc-4e93-8a3e-c4a6b71c6cd4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:25.946778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:25.960291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7679a-47dc-4e93-8a3e-c4a6b71c6cd4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:25.995332 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042daca-29c9-49c4-9992-a1351c0915c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:25.996441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042daca-29c9-49c4-9992-a1351c0915c1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:25.996757 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042daca-29c9-49c4-9992-a1351c0915c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:25.997044 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042daca-29c9-49c4-9992-a1351c0915c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:25.997804 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042daca-29c9-49c4-9992-a1351c0915c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:25.998149 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042daca-29c9-49c4-9992-a1351c0915c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:25.999881 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042daca-29c9-49c4-9992-a1351c0915c1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:26.017259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 536/1069] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:26.018841 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39eafa-c1ee-44c0-9030-86a14dc95916 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:26.123458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39eafa-c1ee-44c0-9030-86a14dc95916 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:26.158440 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a204541-ba5b-45fd-b0bd-95afffa742e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:26.159238 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a204541-ba5b-45fd-b0bd-95afffa742e4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:26.159522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a204541-ba5b-45fd-b0bd-95afffa742e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:26.159799 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a204541-ba5b-45fd-b0bd-95afffa742e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:26.166482 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a204541-ba5b-45fd-b0bd-95afffa742e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:26.167917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a204541-ba5b-45fd-b0bd-95afffa742e4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-1a204541-ba5b-45fd-b0bd-95afffa742e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:52:26.182932 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:26.183917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:26.184883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:52:26.184883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:52:26.184883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:52:26.184883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:52:26.184883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:52:26.184883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:52:26.184883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:26.184883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:26.184883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 534/1070] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:26.192706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042daca-29c9-49c4-9992-a1351c0915c1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:26.295718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042daca-29c9-49c4-9992-a1351c0915c1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:26.331240 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d26f1-9f28-40ac-85d1-0134268af8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:26.331906 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d26f1-9f28-40ac-85d1-0134268af8ae None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:26.332197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d26f1-9f28-40ac-85d1-0134268af8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:26.332502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d26f1-9f28-40ac-85d1-0134268af8ae None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:26.333295 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d26f1-9f28-40ac-85d1-0134268af8ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:26.333613 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d26f1-9f28-40ac-85d1-0134268af8ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:26.335097 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d26f1-9f28-40ac-85d1-0134268af8ae None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:26.352868 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 537/1071] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:52:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:26.354151 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a204541-ba5b-45fd-b0bd-95afffa742e4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:26.380487 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:26.462911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a204541-ba5b-45fd-b0bd-95afffa742e4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:26.502306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd0514-8c36-42b8-ae8a-e9e4a35ae919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:26.503204 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd0514-8c36-42b8-ae8a-e9e4a35ae919 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:26.503489 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd0514-8c36-42b8-ae8a-e9e4a35ae919 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:26.503768 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd0514-8c36-42b8-ae8a-e9e4a35ae919 None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/5368284e42d84e77a6fdb88c2e8dcd3d/roles/e8a21e2686c549e5b16fd9df5f041c72` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:26.506387 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd0514-8c36-42b8-ae8a-e9e4a35ae919 None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=5368284e42d84e77a6fdb88c2e8dcd3d, role_id=e8a21e2686c549e5b16fd9df5f041c72)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:26.507835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd0514-8c36-42b8-ae8a-e9e4a35ae919 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:26.590894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 535/1072] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:52:26 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/5368284e42d84e77a6fdb88c2e8dcd3d/roles/e8a21e2686c549e5b16fd9df5f041c72 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:52:27.505686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df6682-1c69-4092-8e15-fd958203d90b None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:27.506452 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df6682-1c69-4092-8e15-fd958203d90b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:27.506531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df6682-1c69-4092-8e15-fd958203d90b None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:27.508005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 538/1073] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:27.516879 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde106c2-2ef0-4a01-8362-93c21a769d57 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:27.517269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde106c2-2ef0-4a01-8362-93c21a769d57 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:27.517441 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde106c2-2ef0-4a01-8362-93c21a769d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:27.535471 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-cde106c2-2ef0-4a01-8362-93c21a769d57 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:27.536719 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cde106c2-2ef0-4a01-8362-93c21a769d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:27.606657 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 536/1074] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:27.631083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df6682-1c69-4092-8e15-fd958203d90b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:27.642654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:27.742971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df6682-1c69-4092-8e15-fd958203d90b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:27.782678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c34b7-9eaa-4173-a046-e415c4195a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:27.783334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c34b7-9eaa-4173-a046-e415c4195a03 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:27.783651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c34b7-9eaa-4173-a046-e415c4195a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:27.783935 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c34b7-9eaa-4173-a046-e415c4195a03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:27.800794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942cca5-ab8a-4290-8fa4-b190abbd4174 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:27.801324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942cca5-ab8a-4290-8fa4-b190abbd4174 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:27.801609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942cca5-ab8a-4290-8fa4-b190abbd4174 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:27.803538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 537/1075] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:27.809057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8989-ff16-4452-916b-d69e3e60ceb1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:27.809416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8989-ff16-4452-916b-d69e3e60ceb1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:27.809822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c8989-ff16-4452-916b-d69e3e60ceb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:27.828149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-898c8989-ff16-4452-916b-d69e3e60ceb1 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:27.828989 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-898c8989-ff16-4452-916b-d69e3e60ceb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:27.836901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:27.861800 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c34b7-9eaa-4173-a046-e415c4195a03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:27.863262 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c34b7-9eaa-4173-a046-e415c4195a03 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:27.875711 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:27.900547 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 538/1076] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:52:27.924408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8989-ff16-4452-916b-d69e3e60ceb1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:27.978152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 539/1077] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:52:27.984568 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c34b7-9eaa-4173-a046-e415c4195a03 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:28.035175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c8989-ff16-4452-916b-d69e3e60ceb1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:28.074218 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35a082-8d1f-4aa4-bede-b976cf62cbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:28.075090 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35a082-8d1f-4aa4-bede-b976cf62cbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:28.075372 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35a082-8d1f-4aa4-bede-b976cf62cbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:28.075653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35a082-8d1f-4aa4-bede-b976cf62cbc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:28.087368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c34b7-9eaa-4173-a046-e415c4195a03 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:28.087522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35a082-8d1f-4aa4-bede-b976cf62cbc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2a35a082-8d1f-4aa4-bede-b976cf62cbc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:28.095913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:28.096934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:28.097821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 539/1078] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:52:28.099761 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:28.103900 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35a082-8d1f-4aa4-bede-b976cf62cbc1 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:28.106811 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:28.128253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb3f05-7fa9-46fc-977e-58c808978639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:28.129094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb3f05-7fa9-46fc-977e-58c808978639 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:28.129385 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb3f05-7fa9-46fc-977e-58c808978639 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:28.129671 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb3f05-7fa9-46fc-977e-58c808978639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:28.141905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb3f05-7fa9-46fc-977e-58c808978639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:28.143693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb3f05-7fa9-46fc-977e-58c808978639 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a1cb3f05-7fa9-46fc-977e-58c808978639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:28.162605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:28.163563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:28.164779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:28.164779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:28.164779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:28.164779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:28.165505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 540/1079] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:28.172288 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb3f05-7fa9-46fc-977e-58c808978639 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:28.224638 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35a082-8d1f-4aa4-bede-b976cf62cbc1 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:28.263599 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118241d4-f707-4e7e-98bf-6ce5e7de6127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:28.264351 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118241d4-f707-4e7e-98bf-6ce5e7de6127 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:28.265789 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118241d4-f707-4e7e-98bf-6ce5e7de6127 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:28.266056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118241d4-f707-4e7e-98bf-6ce5e7de6127 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:28.266801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118241d4-f707-4e7e-98bf-6ce5e7de6127 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:28.267108 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118241d4-f707-4e7e-98bf-6ce5e7de6127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:28.268615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118241d4-f707-4e7e-98bf-6ce5e7de6127 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:28.278139 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 540/1080] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:28.283182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb3f05-7fa9-46fc-977e-58c808978639 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:28.284919 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118241d4-f707-4e7e-98bf-6ce5e7de6127 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:28.318562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6ece-4ce7-42a7-bd67-46002570ba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:28.319147 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6ece-4ce7-42a7-bd67-46002570ba5c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:28.319439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6ece-4ce7-42a7-bd67-46002570ba5c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:28.319717 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6ece-4ce7-42a7-bd67-46002570ba5c None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:28.320477 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f6ece-4ce7-42a7-bd67-46002570ba5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:28.320794 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f6ece-4ce7-42a7-bd67-46002570ba5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:28.322181 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f6ece-4ce7-42a7-bd67-46002570ba5c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:28.344260 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 541/1081] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:28.364422 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:28.410715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118241d4-f707-4e7e-98bf-6ce5e7de6127 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:28.447248 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74607e5e-f3e2-435e-8d32-45c05699fda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:28.448143 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74607e5e-f3e2-435e-8d32-45c05699fda4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:28.448464 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74607e5e-f3e2-435e-8d32-45c05699fda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:28.448751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74607e5e-f3e2-435e-8d32-45c05699fda4 None admin] PATH_INFO: `/v3/users/b75c329a379c460d8df77a4158d1c83a` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:28.450082 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74607e5e-f3e2-435e-8d32-45c05699fda4 None admin] RBAC: Authorizing `identity:get_user(user_id=b75c329a379c460d8df77a4158d1c83a)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:28.451554 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74607e5e-f3e2-435e-8d32-45c05699fda4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:28.454104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 541/1082] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:28 2022] GET /identity/v3/users/b75c329a379c460d8df77a4158d1c83a => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:28.460141 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6ece-4ce7-42a7-bd67-46002570ba5c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:28.486887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:28.575847 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6ece-4ce7-42a7-bd67-46002570ba5c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:28.614832 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52306ad-9b8a-4569-a507-c436d03a7bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:28.615710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52306ad-9b8a-4569-a507-c436d03a7bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:28.615999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52306ad-9b8a-4569-a507-c436d03a7bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:28.616429 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52306ad-9b8a-4569-a507-c436d03a7bb4 None admin] PATH_INFO: `/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:28.617527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52306ad-9b8a-4569-a507-c436d03a7bb4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a747f9853b646cab08ed44ee8ecbb1e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:28.618985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52306ad-9b8a-4569-a507-c436d03a7bb4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:28.621031 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 542/1083] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:52:28 2022] GET /identity/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:28.628138 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74607e5e-f3e2-435e-8d32-45c05699fda4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:28.739281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74607e5e-f3e2-435e-8d32-45c05699fda4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:28.757411 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d67d20-7bef-4d45-bdc7-65781d80cceb None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:28.757707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d67d20-7bef-4d45-bdc7-65781d80cceb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:28.757988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d67d20-7bef-4d45-bdc7-65781d80cceb None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:28.759680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 543/1084] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:52:28.764357 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09299294-a08d-47b0-9f7e-2f5d77c82529 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:28.765321 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09299294-a08d-47b0-9f7e-2f5d77c82529 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:28.765605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09299294-a08d-47b0-9f7e-2f5d77c82529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:28.769761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:28.774562 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e827eb8-795c-44f2-b09d-bd9bed9c46a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:28.775445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e827eb8-795c-44f2-b09d-bd9bed9c46a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:28.775727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e827eb8-795c-44f2-b09d-bd9bed9c46a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:28.776047 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e827eb8-795c-44f2-b09d-bd9bed9c46a7 None admin] PATH_INFO: `/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e/users/b75c329a379c460d8df77a4158d1c83a/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:28.783380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e827eb8-795c-44f2-b09d-bd9bed9c46a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a747f9853b646cab08ed44ee8ecbb1e, user_id=b75c329a379c460d8df77a4158d1c83a, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:28.783380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e827eb8-795c-44f2-b09d-bd9bed9c46a7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:28.783598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-09299294-a08d-47b0-9f7e-2f5d77c82529 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:28.783598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-09299294-a08d-47b0-9f7e-2f5d77c82529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:28.850117 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 544/1085] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:52:28.870170 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 542/1086] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:52:28 2022] PUT /identity/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e/users/b75c329a379c460d8df77a4158d1c83a/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:52:28.874891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09299294-a08d-47b0-9f7e-2f5d77c82529 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:28.896043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:28.984358 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09299294-a08d-47b0-9f7e-2f5d77c82529 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:29.021389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdef226-3818-4fd2-86a4-29c2ac826830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:29.022256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdef226-3818-4fd2-86a4-29c2ac826830 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:29.022547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdef226-3818-4fd2-86a4-29c2ac826830 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:29.022830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdef226-3818-4fd2-86a4-29c2ac826830 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:29.036496 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdef226-3818-4fd2-86a4-29c2ac826830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-bbdef226-3818-4fd2-86a4-29c2ac826830 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:29.044921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:29.046488 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:29.047370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 545/1087] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 14:52:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:29.053375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e827eb8-795c-44f2-b09d-bd9bed9c46a7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:29.155189 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e827eb8-795c-44f2-b09d-bd9bed9c46a7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:29.189394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ce19b-6d9d-49c7-ad83-b492f78265af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:29.190184 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ce19b-6d9d-49c7-ad83-b492f78265af None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:29.190477 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ce19b-6d9d-49c7-ad83-b492f78265af None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:29.190759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ce19b-6d9d-49c7-ad83-b492f78265af None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:29.191519 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ce19b-6d9d-49c7-ad83-b492f78265af None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:29.191833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ce19b-6d9d-49c7-ad83-b492f78265af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:29.193287 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ce19b-6d9d-49c7-ad83-b492f78265af None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:29.201901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 543/1088] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 14:52:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:29.208058 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdef226-3818-4fd2-86a4-29c2ac826830 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:29.286828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:29.308381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdef226-3818-4fd2-86a4-29c2ac826830 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:29.343502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26320205-1131-499b-9a82-8c1ee6104a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:29.344322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26320205-1131-499b-9a82-8c1ee6104a31 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:29.344612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26320205-1131-499b-9a82-8c1ee6104a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:29.344887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26320205-1131-499b-9a82-8c1ee6104a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:29.355572 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26320205-1131-499b-9a82-8c1ee6104a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:29.357052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26320205-1131-499b-9a82-8c1ee6104a31 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-26320205-1131-499b-9a82-8c1ee6104a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:29.375005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:29.375949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:29.376904 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:29.376904 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:29.376904 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:29.376904 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:29.376904 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 546/1089] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:29.382289 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ce19b-6d9d-49c7-ad83-b492f78265af None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:29.438473 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:29.485201 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ce19b-6d9d-49c7-ad83-b492f78265af None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:29.519998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68a48f-7290-4fb1-bae0-8b8fc5feae9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:29.520736 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68a48f-7290-4fb1-bae0-8b8fc5feae9e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:29.521020 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68a48f-7290-4fb1-bae0-8b8fc5feae9e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:29.521294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68a48f-7290-4fb1-bae0-8b8fc5feae9e None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:29.522011 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68a48f-7290-4fb1-bae0-8b8fc5feae9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:29.522320 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68a48f-7290-4fb1-bae0-8b8fc5feae9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:29.523674 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68a48f-7290-4fb1-bae0-8b8fc5feae9e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:29.536464 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 544/1090] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:29.541819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26320205-1131-499b-9a82-8c1ee6104a31 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:29.642801 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26320205-1131-499b-9a82-8c1ee6104a31 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:29.671628 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:29.677592 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a1148-b69d-475c-82d7-62937c14a7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:29.678382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a1148-b69d-475c-82d7-62937c14a7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:29.678670 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a1148-b69d-475c-82d7-62937c14a7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:29.678950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a1148-b69d-475c-82d7-62937c14a7c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:29.686636 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a1148-b69d-475c-82d7-62937c14a7c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:29.688160 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a1148-b69d-475c-82d7-62937c14a7c9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1d4a1148-b69d-475c-82d7-62937c14a7c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:52:29.702162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:29.703562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:52:29.704824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:29.706839 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 547/1091] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 14:52:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:29.712648 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68a48f-7290-4fb1-bae0-8b8fc5feae9e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:29.811356 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68a48f-7290-4fb1-bae0-8b8fc5feae9e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:29.826665 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:29.847739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a212515-2fba-4682-9606-2dfe40d5449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:29.848441 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a212515-2fba-4682-9606-2dfe40d5449a None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:29.848733 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a212515-2fba-4682-9606-2dfe40d5449a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:29.849016 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a212515-2fba-4682-9606-2dfe40d5449a None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:29.849861 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a212515-2fba-4682-9606-2dfe40d5449a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:29.850183 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a212515-2fba-4682-9606-2dfe40d5449a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:29.851622 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a212515-2fba-4682-9606-2dfe40d5449a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:29.869258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 545/1092] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 14:52:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:29.874971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a1148-b69d-475c-82d7-62937c14a7c9 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:29.977409 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a1148-b69d-475c-82d7-62937c14a7c9 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:30.010716 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306104-7fc2-406a-b7ba-0b51bf06fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:30.011367 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306104-7fc2-406a-b7ba-0b51bf06fc2e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:30.011664 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306104-7fc2-406a-b7ba-0b51bf06fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:30.011948 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306104-7fc2-406a-b7ba-0b51bf06fc2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:30.069976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:30.080878 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3306104-7fc2-406a-b7ba-0b51bf06fc2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:30.089334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3306104-7fc2-406a-b7ba-0b51bf06fc2e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:30.181553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 548/1093] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:52:30.187423 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a212515-2fba-4682-9606-2dfe40d5449a None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:30.288798 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a212515-2fba-4682-9606-2dfe40d5449a None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:30.328007 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6a6a9-155c-4818-86c8-20d1a28acac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:30.328855 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6a6a9-155c-4818-86c8-20d1a28acac5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:30.329140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6a6a9-155c-4818-86c8-20d1a28acac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:30.329420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6a6a9-155c-4818-86c8-20d1a28acac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:30.339268 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6a6a9-155c-4818-86c8-20d1a28acac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:30.340731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6a6a9-155c-4818-86c8-20d1a28acac5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5eb6a6a9-155c-4818-86c8-20d1a28acac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:52:30.358964 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:30.360264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:30.361569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:52:30.361569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:52:30.361569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:52:30.361569 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:30.364370 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 546/1094] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:30.365884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306104-7fc2-406a-b7ba-0b51bf06fc2e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:30.455867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:30.465318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306104-7fc2-406a-b7ba-0b51bf06fc2e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:30.501857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f0931-5340-46af-9137-fe44979e16c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:30.502531 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f0931-5340-46af-9137-fe44979e16c3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:30.502820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f0931-5340-46af-9137-fe44979e16c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:30.503108 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f0931-5340-46af-9137-fe44979e16c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:30.503837 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f0931-5340-46af-9137-fe44979e16c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:30.504148 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f0931-5340-46af-9137-fe44979e16c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:30.505877 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f0931-5340-46af-9137-fe44979e16c3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:30.518777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 549/1095] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:30.524669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6a6a9-155c-4818-86c8-20d1a28acac5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:30.552553 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:30.625521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6a6a9-155c-4818-86c8-20d1a28acac5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:30.658869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7bb17-8cd9-49b6-88a3-ae291188e259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:30.659662 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7bb17-8cd9-49b6-88a3-ae291188e259 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:30.659952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7bb17-8cd9-49b6-88a3-ae291188e259 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:30.660248 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7bb17-8cd9-49b6-88a3-ae291188e259 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:30.666935 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7bb17-8cd9-49b6-88a3-ae291188e259 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:30.668424 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7bb17-8cd9-49b6-88a3-ae291188e259 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-5ce7bb17-8cd9-49b6-88a3-ae291188e259 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:52:30.682812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:30.683960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:30.685050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:52:30.685050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:52:30.685050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:52:30.685050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:52:30.685050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:52:30.685050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:52:30.685050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:52:30.685050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:30.685050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 547/1096] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:52:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:30.690458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f0931-5340-46af-9137-fe44979e16c3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:30.789295 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f0931-5340-46af-9137-fe44979e16c3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:30.822569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bf776-d26b-4586-9f74-17ec59fbc34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:30.823154 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bf776-d26b-4586-9f74-17ec59fbc34b None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:30.823442 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bf776-d26b-4586-9f74-17ec59fbc34b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:30.823725 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bf776-d26b-4586-9f74-17ec59fbc34b None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:30.824547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bf776-d26b-4586-9f74-17ec59fbc34b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:30.824863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bf776-d26b-4586-9f74-17ec59fbc34b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:30.826338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bf776-d26b-4586-9f74-17ec59fbc34b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:30.839396 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 550/1097] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:52:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:52:30.845972 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7bb17-8cd9-49b6-88a3-ae291188e259 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:30.934353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:30.950665 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7bb17-8cd9-49b6-88a3-ae291188e259 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:30.988839 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322289c-ba59-4036-a4db-62ddfcdf86ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:30.989512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322289c-ba59-4036-a4db-62ddfcdf86ea None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:30.989798 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322289c-ba59-4036-a4db-62ddfcdf86ea None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:30.990076 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322289c-ba59-4036-a4db-62ddfcdf86ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:30.990923 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322289c-ba59-4036-a4db-62ddfcdf86ea None admin] RBAC: Adding query filter params (role.id=e8a21e2686c549e5b16fd9df5f041c72, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=5368284e42d84e77a6fdb88c2e8dcd3d) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:30.991238 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322289c-ba59-4036-a4db-62ddfcdf86ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:30.992735 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322289c-ba59-4036-a4db-62ddfcdf86ea None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:31.000353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 548/1098] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:30 2022] GET /identity/v3/role_assignments?user.id=5368284e42d84e77a6fdb88c2e8dcd3d&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=e8a21e2686c549e5b16fd9df5f041c72 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:31.034965 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743e739-c573-4bac-af23-bc5f0d815c65 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:31.035257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743e739-c573-4bac-af23-bc5f0d815c65 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:31.035543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743e739-c573-4bac-af23-bc5f0d815c65 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:31.036976 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 551/1099] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:31.042747 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744099-ff5c-4bca-8f85-b68a2a52bc08 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:31.043045 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744099-ff5c-4bca-8f85-b68a2a52bc08 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:31.043329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744099-ff5c-4bca-8f85-b68a2a52bc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:31.062356 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-59744099-ff5c-4bca-8f85-b68a2a52bc08 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:31.063598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-59744099-ff5c-4bca-8f85-b68a2a52bc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:31.136797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 549/1100] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:31.160974 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743e739-c573-4bac-af23-bc5f0d815c65 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:31.175251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:31.268560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743e739-c573-4bac-af23-bc5f0d815c65 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:31.304107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9e39b-104a-4100-9a65-3f33fef58aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:31.304924 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9e39b-104a-4100-9a65-3f33fef58aad None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:31.305212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9e39b-104a-4100-9a65-3f33fef58aad None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:31.305737 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9e39b-104a-4100-9a65-3f33fef58aad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:31.316184 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9e39b-104a-4100-9a65-3f33fef58aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-ffb9e39b-104a-4100-9a65-3f33fef58aad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:31.323654 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:31.324762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:31.325627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 552/1101] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:52:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:31.331976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59744099-ff5c-4bca-8f85-b68a2a52bc08 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:31.411774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:31.433966 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59744099-ff5c-4bca-8f85-b68a2a52bc08 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:31.471189 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b45c25-b4dc-4396-99c9-798d2e1a2f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:31.471865 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b45c25-b4dc-4396-99c9-798d2e1a2f45 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:31.472156 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b45c25-b4dc-4396-99c9-798d2e1a2f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:31.472612 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b45c25-b4dc-4396-99c9-798d2e1a2f45 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:31.473392 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b45c25-b4dc-4396-99c9-798d2e1a2f45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:31.473702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b45c25-b4dc-4396-99c9-798d2e1a2f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:31.475082 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b45c25-b4dc-4396-99c9-798d2e1a2f45 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:31.483980 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 550/1102] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:52:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:31.490037 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9e39b-104a-4100-9a65-3f33fef58aad None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:31.564922 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:31.597982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9e39b-104a-4100-9a65-3f33fef58aad None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:31.633802 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e865e1d-0b96-4b3b-8ab1-a4209d96c0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:31.634725 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e865e1d-0b96-4b3b-8ab1-a4209d96c0ea None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:31.635020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e865e1d-0b96-4b3b-8ab1-a4209d96c0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:31.635308 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e865e1d-0b96-4b3b-8ab1-a4209d96c0ea None admin] PATH_INFO: `/v3/users/b75c329a379c460d8df77a4158d1c83a` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:31.636529 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e865e1d-0b96-4b3b-8ab1-a4209d96c0ea None admin] RBAC: Authorizing `identity:get_user(user_id=b75c329a379c460d8df77a4158d1c83a)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:31.638057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e865e1d-0b96-4b3b-8ab1-a4209d96c0ea None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:31.640275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 553/1103] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:31 2022] GET /identity/v3/users/b75c329a379c460d8df77a4158d1c83a => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:31.646210 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b45c25-b4dc-4396-99c9-798d2e1a2f45 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:31.752986 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b45c25-b4dc-4396-99c9-798d2e1a2f45 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:31.789407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a11ccd-f742-4177-90e6-ec8f8ea5ddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:31.790068 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a11ccd-f742-4177-90e6-ec8f8ea5ddf7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:31.790359 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a11ccd-f742-4177-90e6-ec8f8ea5ddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:31.790642 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a11ccd-f742-4177-90e6-ec8f8ea5ddf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:31.802348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:31.862599 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a11ccd-f742-4177-90e6-ec8f8ea5ddf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:31.864057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a11ccd-f742-4177-90e6-ec8f8ea5ddf7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:31.965732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 551/1104] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:52:31.972047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e865e1d-0b96-4b3b-8ab1-a4209d96c0ea None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:32.071102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e865e1d-0b96-4b3b-8ab1-a4209d96c0ea None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:32.104609 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818d1f0-c0eb-47e0-b475-09eb7711278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:32.105459 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818d1f0-c0eb-47e0-b475-09eb7711278e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:32.105751 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818d1f0-c0eb-47e0-b475-09eb7711278e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:32.106032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818d1f0-c0eb-47e0-b475-09eb7711278e None admin] PATH_INFO: `/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:32.107201 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818d1f0-c0eb-47e0-b475-09eb7711278e None admin] RBAC: Authorizing `identity:get_project(project_id=2a747f9853b646cab08ed44ee8ecbb1e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:32.108798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818d1f0-c0eb-47e0-b475-09eb7711278e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:32.110490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 554/1105] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:52:31 2022] GET /identity/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:32.116596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a11ccd-f742-4177-90e6-ec8f8ea5ddf7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:32.157447 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:32.213471 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a11ccd-f742-4177-90e6-ec8f8ea5ddf7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:32.247140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3d6bd-c810-4926-a6b3-c457c228c0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:32.247786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3d6bd-c810-4926-a6b3-c457c228c0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:32.248080 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3d6bd-c810-4926-a6b3-c457c228c0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:32.248400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3d6bd-c810-4926-a6b3-c457c228c0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:32.249363 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3d6bd-c810-4926-a6b3-c457c228c0c6 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=2a747f9853b646cab08ed44ee8ecbb1e, user.id=b75c329a379c460d8df77a4158d1c83a) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:32.249680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3d6bd-c810-4926-a6b3-c457c228c0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:32.251101 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3d6bd-c810-4926-a6b3-c457c228c0c6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:32.257125 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 552/1106] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:32 2022] GET /identity/v3/role_assignments?user.id=b75c329a379c460d8df77a4158d1c83a&scope.project.id=2a747f9853b646cab08ed44ee8ecbb1e&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:34.413277 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3027-2842-4df3-908e-c12e88c66dfc None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:34.413819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3027-2842-4df3-908e-c12e88c66dfc None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:34.413819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f3027-2842-4df3-908e-c12e88c66dfc None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:34.415200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 555/1107] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:34.421939 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fc232-ba11-4383-bd2b-90f1de0325f8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:34.422383 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fc232-ba11-4383-bd2b-90f1de0325f8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:34.422600 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fc232-ba11-4383-bd2b-90f1de0325f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:34.439682 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a82fc232-ba11-4383-bd2b-90f1de0325f8 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:34.440895 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a82fc232-ba11-4383-bd2b-90f1de0325f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:34.509711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 553/1108] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:34.534243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f3027-2842-4df3-908e-c12e88c66dfc None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:34.543722 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:34.637866 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f3027-2842-4df3-908e-c12e88c66dfc None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:34.671934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb139cde-4dc4-459c-a18e-f709c744d144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:34.672680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb139cde-4dc4-459c-a18e-f709c744d144 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:34.672971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb139cde-4dc4-459c-a18e-f709c744d144 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:34.673254 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb139cde-4dc4-459c-a18e-f709c744d144 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:34.674432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb139cde-4dc4-459c-a18e-f709c744d144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:34.675912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb139cde-4dc4-459c-a18e-f709c744d144 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:34.677651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 556/1109] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:34.683427 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fc232-ba11-4383-bd2b-90f1de0325f8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:34.715798 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:34.781598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fc232-ba11-4383-bd2b-90f1de0325f8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:34.819336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28177381-f2bd-4a06-83c0-33fde4d723d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:34.820040 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177381-f2bd-4a06-83c0-33fde4d723d9 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:34.820435 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177381-f2bd-4a06-83c0-33fde4d723d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:34.820720 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177381-f2bd-4a06-83c0-33fde4d723d9 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:34.821444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28177381-f2bd-4a06-83c0-33fde4d723d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:34.822892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28177381-f2bd-4a06-83c0-33fde4d723d9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:34.918594 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 554/1110] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:52:34 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:52:37.190560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf1b73-4b03-4eea-a699-6c189dba69b0 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:37.191130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf1b73-4b03-4eea-a699-6c189dba69b0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:37.191130 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf1b73-4b03-4eea-a699-6c189dba69b0 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:37.192530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 557/1111] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:37.197691 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f5262-96e2-4edd-a97e-492e7d7ecb95 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:37.198082 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f5262-96e2-4edd-a97e-492e7d7ecb95 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:37.198259 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f5262-96e2-4edd-a97e-492e7d7ecb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:37.215035 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0e3f5262-96e2-4edd-a97e-492e7d7ecb95 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:37.216240 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0e3f5262-96e2-4edd-a97e-492e7d7ecb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:37.284550 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 555/1112] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:37.308444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf1b73-4b03-4eea-a699-6c189dba69b0 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:37.386012 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:37.411853 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf1b73-4b03-4eea-a699-6c189dba69b0 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:37.444034 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6911c0-13af-40ad-a5af-b0f1baa714c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:37.444837 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6911c0-13af-40ad-a5af-b0f1baa714c3 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:37.445127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6911c0-13af-40ad-a5af-b0f1baa714c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:37.445450 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6911c0-13af-40ad-a5af-b0f1baa714c3 None admin] PATH_INFO: `/v3/roles/f4c6e6a3b05a42abb56447797f7ade15` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:37.446880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6911c0-13af-40ad-a5af-b0f1baa714c3 None admin] RBAC: Authorizing `identity:get_role(role_id=f4c6e6a3b05a42abb56447797f7ade15)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:37.448269 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6911c0-13af-40ad-a5af-b0f1baa714c3 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:37.449646 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 558/1113] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:37 2022] GET /identity/v3/roles/f4c6e6a3b05a42abb56447797f7ade15 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:37.456008 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f5262-96e2-4edd-a97e-492e7d7ecb95 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:37.477344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:37.554997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f5262-96e2-4edd-a97e-492e7d7ecb95 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:37.588527 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9532a-5385-44fb-8055-b5fb5c2203ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:37.589382 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9532a-5385-44fb-8055-b5fb5c2203ca None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:37.589668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9532a-5385-44fb-8055-b5fb5c2203ca None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:37.590018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9532a-5385-44fb-8055-b5fb5c2203ca None admin] PATH_INFO: `/v3/users/dd4b8016f9574bc29c38e95460682500` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:37.604044 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9532a-5385-44fb-8055-b5fb5c2203ca None admin] RBAC: Authorizing `identity:get_user(user_id=dd4b8016f9574bc29c38e95460682500)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:37.605568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9532a-5385-44fb-8055-b5fb5c2203ca None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:37.607219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 556/1114] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:37 2022] GET /identity/v3/users/dd4b8016f9574bc29c38e95460682500 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:37.613090 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6911c0-13af-40ad-a5af-b0f1baa714c3 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:37.714109 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6911c0-13af-40ad-a5af-b0f1baa714c3 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:37.746971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a093fa-6f49-4a0f-b67e-d84f40a66f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:37.747615 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a093fa-6f49-4a0f-b67e-d84f40a66f45 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:37.747967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a093fa-6f49-4a0f-b67e-d84f40a66f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:37.748831 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a093fa-6f49-4a0f-b67e-d84f40a66f45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:37.760845 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:37.815467 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a093fa-6f49-4a0f-b67e-d84f40a66f45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:37.816944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a093fa-6f49-4a0f-b67e-d84f40a66f45 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:37.915230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 559/1115] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:52:37.921578 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9532a-5385-44fb-8055-b5fb5c2203ca None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:37.999819 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:38.020559 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9532a-5385-44fb-8055-b5fb5c2203ca None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:38.054115 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60f777-873a-4d73-9eaa-05508e7af6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:38.054962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60f777-873a-4d73-9eaa-05508e7af6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:38.055245 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60f777-873a-4d73-9eaa-05508e7af6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:38.055523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60f777-873a-4d73-9eaa-05508e7af6b4 None admin] PATH_INFO: `/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:38.056735 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60f777-873a-4d73-9eaa-05508e7af6b4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a747f9853b646cab08ed44ee8ecbb1e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:38.058220 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60f777-873a-4d73-9eaa-05508e7af6b4 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:38.059905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 557/1116] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:52:37 2022] GET /identity/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:38.066857 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a093fa-6f49-4a0f-b67e-d84f40a66f45 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:38.109020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:38.163869 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a093fa-6f49-4a0f-b67e-d84f40a66f45 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:38.197705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176d317-bc10-4d78-86f8-a807e2436d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:38.198351 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176d317-bc10-4d78-86f8-a807e2436d10 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:38.198631 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176d317-bc10-4d78-86f8-a807e2436d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:38.198934 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176d317-bc10-4d78-86f8-a807e2436d10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:38.199769 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176d317-bc10-4d78-86f8-a807e2436d10 None admin] RBAC: Adding query filter params (role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=2a747f9853b646cab08ed44ee8ecbb1e, user.id=dd4b8016f9574bc29c38e95460682500) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:38.200088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176d317-bc10-4d78-86f8-a807e2436d10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:38.201603 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176d317-bc10-4d78-86f8-a807e2436d10 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:38.207193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 560/1117] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:38 2022] GET /identity/v3/role_assignments?user.id=dd4b8016f9574bc29c38e95460682500&scope.project.id=2a747f9853b646cab08ed44ee8ecbb1e&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:40.320041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab39e-82b3-4a31-8936-93cd5b96b9ce None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:40.320705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab39e-82b3-4a31-8936-93cd5b96b9ce None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:40.320705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab39e-82b3-4a31-8936-93cd5b96b9ce None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:40.322002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 558/1118] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:40.327814 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930644e4-c64c-4c37-b2e1-49621307e616 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:40.328254 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930644e4-c64c-4c37-b2e1-49621307e616 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:40.328410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930644e4-c64c-4c37-b2e1-49621307e616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:40.345749 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-930644e4-c64c-4c37-b2e1-49621307e616 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:40.346932 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-930644e4-c64c-4c37-b2e1-49621307e616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:40.414744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 561/1119] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:40.438692 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ab39e-82b3-4a31-8936-93cd5b96b9ce None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:40.539588 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ab39e-82b3-4a31-8936-93cd5b96b9ce None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:40.573472 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2e084-e8ec-45ee-b338-9c6b989fb001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:40.574251 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2e084-e8ec-45ee-b338-9c6b989fb001 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:40.574534 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2e084-e8ec-45ee-b338-9c6b989fb001 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:40.574812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2e084-e8ec-45ee-b338-9c6b989fb001 None admin] PATH_INFO: `/v3/roles/f4c6e6a3b05a42abb56447797f7ade15` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:40.576364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2e084-e8ec-45ee-b338-9c6b989fb001 None admin] RBAC: Authorizing `identity:get_role(role_id=f4c6e6a3b05a42abb56447797f7ade15)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:40.577723 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2e084-e8ec-45ee-b338-9c6b989fb001 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:40.579085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 559/1120] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:40 2022] GET /identity/v3/roles/f4c6e6a3b05a42abb56447797f7ade15 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:40.585140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930644e4-c64c-4c37-b2e1-49621307e616 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:40.621434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:40.688602 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930644e4-c64c-4c37-b2e1-49621307e616 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:40.721493 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbf607-a636-4fb2-b07b-6a2dfeb84dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:40.722346 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf607-a636-4fb2-b07b-6a2dfeb84dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:40.722634 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf607-a636-4fb2-b07b-6a2dfeb84dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:40.722911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbf607-a636-4fb2-b07b-6a2dfeb84dd1 None admin] PATH_INFO: `/v3/users/dd4b8016f9574bc29c38e95460682500` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:40.724078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dbf607-a636-4fb2-b07b-6a2dfeb84dd1 None admin] RBAC: Authorizing `identity:get_user(user_id=dd4b8016f9574bc29c38e95460682500)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:40.725547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dbf607-a636-4fb2-b07b-6a2dfeb84dd1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:40.727169 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 562/1121] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:40 2022] GET /identity/v3/users/dd4b8016f9574bc29c38e95460682500 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:40.733769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2e084-e8ec-45ee-b338-9c6b989fb001 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:40.749720 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:40.831532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2e084-e8ec-45ee-b338-9c6b989fb001 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:40.865185 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19188061-38ed-4083-b1a6-8241dfb54d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:40.866037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19188061-38ed-4083-b1a6-8241dfb54d88 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:40.866328 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19188061-38ed-4083-b1a6-8241dfb54d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:40.866607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19188061-38ed-4083-b1a6-8241dfb54d88 None admin] PATH_INFO: `/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:40.867781 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19188061-38ed-4083-b1a6-8241dfb54d88 None admin] RBAC: Authorizing `identity:get_project(project_id=2a747f9853b646cab08ed44ee8ecbb1e)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:40.869316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19188061-38ed-4083-b1a6-8241dfb54d88 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:40.870993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 560/1122] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:52:40 2022] GET /identity/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:40.877039 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbf607-a636-4fb2-b07b-6a2dfeb84dd1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:40.974970 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbf607-a636-4fb2-b07b-6a2dfeb84dd1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:40.977314 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:41.008799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f225549-fb3b-4236-bb06-a7931ab5f3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:41.009686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225549-fb3b-4236-bb06-a7931ab5f3fa None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:41.009971 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225549-fb3b-4236-bb06-a7931ab5f3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:41.010247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225549-fb3b-4236-bb06-a7931ab5f3fa None admin] PATH_INFO: `/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e/users/dd4b8016f9574bc29c38e95460682500/roles/f4c6e6a3b05a42abb56447797f7ade15` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:41.012530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f225549-fb3b-4236-bb06-a7931ab5f3fa None admin] RBAC: Authorizing `identity:create_grant(project_id=2a747f9853b646cab08ed44ee8ecbb1e, user_id=dd4b8016f9574bc29c38e95460682500, role_id=f4c6e6a3b05a42abb56447797f7ade15)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:41.014706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f225549-fb3b-4236-bb06-a7931ab5f3fa None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:41.090225 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 563/1123] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:52:40 2022] PUT /identity/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e/users/dd4b8016f9574bc29c38e95460682500/roles/f4c6e6a3b05a42abb56447797f7ade15 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:52:43.234129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538c263-0893-4fca-a00a-9d605c39423e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:43.234680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538c263-0893-4fca-a00a-9d605c39423e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:43.234680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538c263-0893-4fca-a00a-9d605c39423e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:43.236060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 561/1124] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:43.241604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b16303-f0b3-42ba-989a-13466c37ebae None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:43.242079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b16303-f0b3-42ba-989a-13466c37ebae None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:43.242175 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b16303-f0b3-42ba-989a-13466c37ebae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:43.258567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-69b16303-f0b3-42ba-989a-13466c37ebae None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:43.259748 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-69b16303-f0b3-42ba-989a-13466c37ebae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:43.329300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 564/1125] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:52:43.353246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538c263-0893-4fca-a00a-9d605c39423e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:43.436122 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:43.454910 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538c263-0893-4fca-a00a-9d605c39423e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:43.488437 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9d83-f945-4e45-8eaa-8f0b496e9e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:43.489215 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9d83-f945-4e45-8eaa-8f0b496e9e9f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:43.489498 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9d83-f945-4e45-8eaa-8f0b496e9e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:43.489772 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9d83-f945-4e45-8eaa-8f0b496e9e9f None admin] PATH_INFO: `/v3/roles/f4c6e6a3b05a42abb56447797f7ade15` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:43.491234 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd9d83-f945-4e45-8eaa-8f0b496e9e9f None admin] RBAC: Authorizing `identity:get_role(role_id=f4c6e6a3b05a42abb56447797f7ade15)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:43.492715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd9d83-f945-4e45-8eaa-8f0b496e9e9f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:43.494084 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 562/1126] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:43 2022] GET /identity/v3/roles/f4c6e6a3b05a42abb56447797f7ade15 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:43.499990 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b16303-f0b3-42ba-989a-13466c37ebae None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:43.593000 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:43.599534 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b16303-f0b3-42ba-989a-13466c37ebae None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:43.633642 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff9b4-ea6f-4bdd-93cb-3f840ad0b207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:43.634506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff9b4-ea6f-4bdd-93cb-3f840ad0b207 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:43.634791 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff9b4-ea6f-4bdd-93cb-3f840ad0b207 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:43.635068 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff9b4-ea6f-4bdd-93cb-3f840ad0b207 None admin] PATH_INFO: `/v3/users/dd4b8016f9574bc29c38e95460682500` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:43.636274 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ff9b4-ea6f-4bdd-93cb-3f840ad0b207 None admin] RBAC: Authorizing `identity:get_user(user_id=dd4b8016f9574bc29c38e95460682500)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:43.637725 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ff9b4-ea6f-4bdd-93cb-3f840ad0b207 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:43.640367 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 565/1127] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:43 2022] GET /identity/v3/users/dd4b8016f9574bc29c38e95460682500 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:43.646133 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9d83-f945-4e45-8eaa-8f0b496e9e9f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:43.745411 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9d83-f945-4e45-8eaa-8f0b496e9e9f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:43.778913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa817ba6-7d86-4f8f-8920-97bbdd5ed47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:43.780167 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa817ba6-7d86-4f8f-8920-97bbdd5ed47c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:43.780485 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa817ba6-7d86-4f8f-8920-97bbdd5ed47c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:43.780769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa817ba6-7d86-4f8f-8920-97bbdd5ed47c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:43.802609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:43.849748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa817ba6-7d86-4f8f-8920-97bbdd5ed47c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:43.851189 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa817ba6-7d86-4f8f-8920-97bbdd5ed47c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:43.947037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 563/1128] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:52:43.953740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff9b4-ea6f-4bdd-93cb-3f840ad0b207 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:44.051832 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff9b4-ea6f-4bdd-93cb-3f840ad0b207 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:44.087109 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1d03c-5a2b-4a98-9549-a179ad64b352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:44.087961 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1d03c-5a2b-4a98-9549-a179ad64b352 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:44.088268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1d03c-5a2b-4a98-9549-a179ad64b352 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:44.088552 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1d03c-5a2b-4a98-9549-a179ad64b352 None admin] PATH_INFO: `/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:44.089741 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1d03c-5a2b-4a98-9549-a179ad64b352 None admin] RBAC: Authorizing `identity:get_project(project_id=2a747f9853b646cab08ed44ee8ecbb1e)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:44.091166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1d03c-5a2b-4a98-9549-a179ad64b352 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:44.094190 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 566/1129] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:52:43 2022] GET /identity/v3/projects/2a747f9853b646cab08ed44ee8ecbb1e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:44.100348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa817ba6-7d86-4f8f-8920-97bbdd5ed47c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:44.149492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:44.197701 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa817ba6-7d86-4f8f-8920-97bbdd5ed47c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:44.231979 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb03a0-3f8d-4008-9a19-36bd9a571f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:44.232769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb03a0-3f8d-4008-9a19-36bd9a571f4f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:44.233075 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb03a0-3f8d-4008-9a19-36bd9a571f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:44.233362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb03a0-3f8d-4008-9a19-36bd9a571f4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:44.234195 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb03a0-3f8d-4008-9a19-36bd9a571f4f None admin] RBAC: Adding query filter params (role.id=f4c6e6a3b05a42abb56447797f7ade15, scope.project.id=2a747f9853b646cab08ed44ee8ecbb1e, user.id=dd4b8016f9574bc29c38e95460682500) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:44.234749 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb03a0-3f8d-4008-9a19-36bd9a571f4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:44.235976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb03a0-3f8d-4008-9a19-36bd9a571f4f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:44.241791 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 564/1130] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:44 2022] GET /identity/v3/role_assignments?user.id=dd4b8016f9574bc29c38e95460682500&scope.project.id=2a747f9853b646cab08ed44ee8ecbb1e&role.id=f4c6e6a3b05a42abb56447797f7ade15 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:46.389898 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7084c-b442-4ec0-9113-d7d46558a402 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:46.390462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7084c-b442-4ec0-9113-d7d46558a402 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:46.390462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7084c-b442-4ec0-9113-d7d46558a402 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:46.391865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 567/1131] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:46.396954 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c1ae9-3906-48ff-88fb-09ec1b921556 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:46.397377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c1ae9-3906-48ff-88fb-09ec1b921556 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:46.397550 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c1ae9-3906-48ff-88fb-09ec1b921556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:46.415326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-116c1ae9-3906-48ff-88fb-09ec1b921556 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:46.422848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-116c1ae9-3906-48ff-88fb-09ec1b921556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:46.488730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 565/1132] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:46.512364 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7084c-b442-4ec0-9113-d7d46558a402 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:46.543079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:46.615133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7084c-b442-4ec0-9113-d7d46558a402 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:46.649166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef1dca-f466-4ce6-a60c-cd99a9943f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:46.649873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef1dca-f466-4ce6-a60c-cd99a9943f7e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:46.650158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef1dca-f466-4ce6-a60c-cd99a9943f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:46.650434 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef1dca-f466-4ce6-a60c-cd99a9943f7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:46.651587 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef1dca-f466-4ce6-a60c-cd99a9943f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:46.653040 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef1dca-f466-4ce6-a60c-cd99a9943f7e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:46.654985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 568/1133] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:46.660883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c1ae9-3906-48ff-88fb-09ec1b921556 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:46.706635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:46.759486 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c1ae9-3906-48ff-88fb-09ec1b921556 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:46.793504 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c2db8-749f-44cd-8081-81b0e126f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:46.794158 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c2db8-749f-44cd-8081-81b0e126f28a None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:46.794467 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c2db8-749f-44cd-8081-81b0e126f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:46.795431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c2db8-749f-44cd-8081-81b0e126f28a None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:46.795563 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c2db8-749f-44cd-8081-81b0e126f28a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:46.797053 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c2db8-749f-44cd-8081-81b0e126f28a None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:46.887238 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 566/1134] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:52:46 2022] POST /identity/v3/projects => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:49.082828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22240-79c4-4bee-9d4a-4fd7b28fa3f4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:49.083438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22240-79c4-4bee-9d4a-4fd7b28fa3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:49.083438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22240-79c4-4bee-9d4a-4fd7b28fa3f4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:49.084942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 569/1135] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:49.090344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634bc88-d2df-4bd3-b998-d1aabcb09282 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:49.090768 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634bc88-d2df-4bd3-b998-d1aabcb09282 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:49.090913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634bc88-d2df-4bd3-b998-d1aabcb09282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:49.108405 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a634bc88-d2df-4bd3-b998-d1aabcb09282 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:49.109587 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a634bc88-d2df-4bd3-b998-d1aabcb09282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:49.174476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 567/1136] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:49.198949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c22240-79c4-4bee-9d4a-4fd7b28fa3f4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:49.294956 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:49.299610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c22240-79c4-4bee-9d4a-4fd7b28fa3f4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:49.333645 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a5911-6ceb-42c7-bc0f-c4c54efe738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:49.334355 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5911-6ceb-42c7-bc0f-c4c54efe738c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:49.334640 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5911-6ceb-42c7-bc0f-c4c54efe738c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:49.334917 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5911-6ceb-42c7-bc0f-c4c54efe738c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:49.336158 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a5911-6ceb-42c7-bc0f-c4c54efe738c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:49.337614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a5911-6ceb-42c7-bc0f-c4c54efe738c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:49.339259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 570/1137] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:52:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:49.345786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634bc88-d2df-4bd3-b998-d1aabcb09282 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:49.376960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:49.451705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634bc88-d2df-4bd3-b998-d1aabcb09282 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:49.485653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1ebbc-ba60-4d3f-84ac-472627b2eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:49.486333 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ebbc-ba60-4d3f-84ac-472627b2eb64 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:49.486731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ebbc-ba60-4d3f-84ac-472627b2eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:49.487020 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ebbc-ba60-4d3f-84ac-472627b2eb64 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:49.487760 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ebbc-ba60-4d3f-84ac-472627b2eb64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:49.489223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ebbc-ba60-4d3f-84ac-472627b2eb64 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:49.566165 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 568/1138] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:52:49 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:52:51.757920 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35330e33-3c53-4bb0-a5b2-5315ee99ac5d None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:51.758457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35330e33-3c53-4bb0-a5b2-5315ee99ac5d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:51.758457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35330e33-3c53-4bb0-a5b2-5315ee99ac5d None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:51.759861 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 571/1139] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:51.765881 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276a046-5b7d-4000-96fb-c7fb960388df None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:51.766339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276a046-5b7d-4000-96fb-c7fb960388df None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:51.766433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276a046-5b7d-4000-96fb-c7fb960388df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:51.783743 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a276a046-5b7d-4000-96fb-c7fb960388df None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:51.784963 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a276a046-5b7d-4000-96fb-c7fb960388df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:51.850572 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 569/1140] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:51.875350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35330e33-3c53-4bb0-a5b2-5315ee99ac5d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:51.974332 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35330e33-3c53-4bb0-a5b2-5315ee99ac5d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:52.007706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0eb1c5-0943-4323-9480-30616cb21f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:52.008511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0eb1c5-0943-4323-9480-30616cb21f9d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:52.008801 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0eb1c5-0943-4323-9480-30616cb21f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:52.009082 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0eb1c5-0943-4323-9480-30616cb21f9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:52.020243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0eb1c5-0943-4323-9480-30616cb21f9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-8a0eb1c5-0943-4323-9480-30616cb21f9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:52.028038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:52.029204 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:52.030268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 572/1141] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:52:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:52.035367 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276a046-5b7d-4000-96fb-c7fb960388df None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:52.049459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:52.132990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276a046-5b7d-4000-96fb-c7fb960388df None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:52.168018 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb53b2-407a-4b77-8d45-70904782c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:52.168707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb53b2-407a-4b77-8d45-70904782c28f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:52.168996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb53b2-407a-4b77-8d45-70904782c28f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:52.169449 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb53b2-407a-4b77-8d45-70904782c28f None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:52.170334 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb53b2-407a-4b77-8d45-70904782c28f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:52.170647 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb53b2-407a-4b77-8d45-70904782c28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:52.172012 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb53b2-407a-4b77-8d45-70904782c28f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:52.179929 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 570/1142] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:52:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:52.186487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0eb1c5-0943-4323-9480-30616cb21f9d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:52.215226 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:52.288086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0eb1c5-0943-4323-9480-30616cb21f9d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:52.322355 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827894d-abcd-4829-b9b3-bfe452846a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:52.323221 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827894d-abcd-4829-b9b3-bfe452846a0f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:52.323512 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827894d-abcd-4829-b9b3-bfe452846a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:52.323793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827894d-abcd-4829-b9b3-bfe452846a0f None admin] PATH_INFO: `/v3/users/f50f5a12f5054c9e9fc2d3d6026842c4` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:52.337271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827894d-abcd-4829-b9b3-bfe452846a0f None admin] RBAC: Authorizing `identity:get_user(user_id=f50f5a12f5054c9e9fc2d3d6026842c4)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:52.338720 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827894d-abcd-4829-b9b3-bfe452846a0f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:52.340614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 573/1143] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:52 2022] GET /identity/v3/users/f50f5a12f5054c9e9fc2d3d6026842c4 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:52.346724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb53b2-407a-4b77-8d45-70904782c28f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:52.421324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:52.445937 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb53b2-407a-4b77-8d45-70904782c28f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:52.480412 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48253048-037b-47f3-a2a2-2767d2bdbe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:52.481052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48253048-037b-47f3-a2a2-2767d2bdbe50 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:52.481336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48253048-037b-47f3-a2a2-2767d2bdbe50 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:52.482120 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48253048-037b-47f3-a2a2-2767d2bdbe50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:52.550676 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48253048-037b-47f3-a2a2-2767d2bdbe50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:52.550676 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48253048-037b-47f3-a2a2-2767d2bdbe50 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:52.621092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:52.646494 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 571/1144] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:52:52.652401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827894d-abcd-4829-b9b3-bfe452846a0f None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:52.735038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:52.750435 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827894d-abcd-4829-b9b3-bfe452846a0f None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:52.783884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146d366-b50d-43f7-9e6f-5dce6e76f92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:52.784770 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146d366-b50d-43f7-9e6f-5dce6e76f92d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:52.785061 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146d366-b50d-43f7-9e6f-5dce6e76f92d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:52.785345 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146d366-b50d-43f7-9e6f-5dce6e76f92d None admin] PATH_INFO: `/v3/projects/83f4130a59674569965f400f1349f55c` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:52.786540 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146d366-b50d-43f7-9e6f-5dce6e76f92d None admin] RBAC: Authorizing `identity:get_project(project_id=83f4130a59674569965f400f1349f55c)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:52.788015 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146d366-b50d-43f7-9e6f-5dce6e76f92d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:52.789774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 574/1145] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:52:52 2022] GET /identity/v3/projects/83f4130a59674569965f400f1349f55c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:52.796952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48253048-037b-47f3-a2a2-2767d2bdbe50 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:52.901704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48253048-037b-47f3-a2a2-2767d2bdbe50 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:52.935765 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7821e1b-80b1-4ddc-aad0-8dfea0627720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:52.936475 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7821e1b-80b1-4ddc-aad0-8dfea0627720 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:52.936767 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7821e1b-80b1-4ddc-aad0-8dfea0627720 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:52.937050 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7821e1b-80b1-4ddc-aad0-8dfea0627720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:52.937953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7821e1b-80b1-4ddc-aad0-8dfea0627720 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=83f4130a59674569965f400f1349f55c, user.id=f50f5a12f5054c9e9fc2d3d6026842c4) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:52.938300 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7821e1b-80b1-4ddc-aad0-8dfea0627720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:52.939727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7821e1b-80b1-4ddc-aad0-8dfea0627720 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:52.945191 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 572/1146] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:52 2022] GET /identity/v3/role_assignments?user.id=f50f5a12f5054c9e9fc2d3d6026842c4&scope.project.id=83f4130a59674569965f400f1349f55c&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:55.066182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365c07d-6d40-4eeb-9305-093c953fee10 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:55.066724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365c07d-6d40-4eeb-9305-093c953fee10 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:55.066724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365c07d-6d40-4eeb-9305-093c953fee10 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:55.068110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 575/1147] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:55.073155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7e01a-c437-4a17-a2ec-883d36a05086 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:55.073548 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7e01a-c437-4a17-a2ec-883d36a05086 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:55.073723 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7e01a-c437-4a17-a2ec-883d36a05086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:55.090415 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-6be7e01a-c437-4a17-a2ec-883d36a05086 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:55.092752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-6be7e01a-c437-4a17-a2ec-883d36a05086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:55.122855 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:55.158604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 573/1148] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:55.181924 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365c07d-6d40-4eeb-9305-093c953fee10 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:55.286334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365c07d-6d40-4eeb-9305-093c953fee10 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:55.320315 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321cb5f-a5eb-4ccb-92d7-b6168117dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:55.321056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321cb5f-a5eb-4ccb-92d7-b6168117dbde None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:55.321370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321cb5f-a5eb-4ccb-92d7-b6168117dbde None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:55.321672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321cb5f-a5eb-4ccb-92d7-b6168117dbde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:55.332499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321cb5f-a5eb-4ccb-92d7-b6168117dbde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-c321cb5f-a5eb-4ccb-92d7-b6168117dbde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:55.340054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:55.341925 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:52:55.343151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 576/1149] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:52:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:55.348425 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7e01a-c437-4a17-a2ec-883d36a05086 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:55.447034 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7e01a-c437-4a17-a2ec-883d36a05086 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:55.480868 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3a969-5c4f-49ea-97aa-8a45da875168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:55.481565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3a969-5c4f-49ea-97aa-8a45da875168 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:55.481855 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3a969-5c4f-49ea-97aa-8a45da875168 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:55.482133 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3a969-5c4f-49ea-97aa-8a45da875168 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:55.482906 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3a969-5c4f-49ea-97aa-8a45da875168 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:55.483187 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3a969-5c4f-49ea-97aa-8a45da875168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:55.484571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3a969-5c4f-49ea-97aa-8a45da875168 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:55.492380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 574/1150] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:52:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:55.498421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321cb5f-a5eb-4ccb-92d7-b6168117dbde None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:55.514505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:55.598818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321cb5f-a5eb-4ccb-92d7-b6168117dbde None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:55.631257 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbf8d6-cd2a-4368-b75e-ef2d9b07da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:55.632113 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbf8d6-cd2a-4368-b75e-ef2d9b07da68 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:55.632425 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbf8d6-cd2a-4368-b75e-ef2d9b07da68 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:55.634400 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbf8d6-cd2a-4368-b75e-ef2d9b07da68 None admin] PATH_INFO: `/v3/users/f50f5a12f5054c9e9fc2d3d6026842c4` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:55.634400 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbf8d6-cd2a-4368-b75e-ef2d9b07da68 None admin] RBAC: Authorizing `identity:get_user(user_id=f50f5a12f5054c9e9fc2d3d6026842c4)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:55.635281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbf8d6-cd2a-4368-b75e-ef2d9b07da68 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:55.636921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 577/1151] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:55 2022] GET /identity/v3/users/f50f5a12f5054c9e9fc2d3d6026842c4 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:55.643482 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3a969-5c4f-49ea-97aa-8a45da875168 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:55.673973 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:55.743003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3a969-5c4f-49ea-97aa-8a45da875168 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:55.777350 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0c3d-adc8-4af4-aed2-1984dd95cea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:55.778192 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0c3d-adc8-4af4-aed2-1984dd95cea3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:55.779006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0c3d-adc8-4af4-aed2-1984dd95cea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:55.779294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0c3d-adc8-4af4-aed2-1984dd95cea3 None admin] PATH_INFO: `/v3/projects/83f4130a59674569965f400f1349f55c` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:55.780492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e0c3d-adc8-4af4-aed2-1984dd95cea3 None admin] RBAC: Authorizing `identity:get_project(project_id=83f4130a59674569965f400f1349f55c)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:55.781920 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e0c3d-adc8-4af4-aed2-1984dd95cea3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:55.783587 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 575/1152] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:52:55 2022] GET /identity/v3/projects/83f4130a59674569965f400f1349f55c => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:55.789635 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbf8d6-cd2a-4368-b75e-ef2d9b07da68 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:55.869545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:55.889831 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbf8d6-cd2a-4368-b75e-ef2d9b07da68 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:55.922710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2840ba-4331-4ed1-a8ea-204b7d33b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:55.923610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2840ba-4331-4ed1-a8ea-204b7d33b740 None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:55.923905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2840ba-4331-4ed1-a8ea-204b7d33b740 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:55.924186 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2840ba-4331-4ed1-a8ea-204b7d33b740 None admin] PATH_INFO: `/v3/projects/83f4130a59674569965f400f1349f55c/users/f50f5a12f5054c9e9fc2d3d6026842c4/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:55.926157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2840ba-4331-4ed1-a8ea-204b7d33b740 None admin] RBAC: Authorizing `identity:create_grant(project_id=83f4130a59674569965f400f1349f55c, user_id=f50f5a12f5054c9e9fc2d3d6026842c4, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:55.927621 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2840ba-4331-4ed1-a8ea-204b7d33b740 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:56.022582 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 578/1153] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:52:55 2022] PUT /identity/v3/projects/83f4130a59674569965f400f1349f55c/users/f50f5a12f5054c9e9fc2d3d6026842c4/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:52:58.138832 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03bd5d-6e02-463e-ab5d-7624d2b34237 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:58.139377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03bd5d-6e02-463e-ab5d-7624d2b34237 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:58.139377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03bd5d-6e02-463e-ab5d-7624d2b34237 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:58.140945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 576/1154] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:52:58.146397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b562629-5eea-4b5c-8ed3-9d29dbff6c45 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:58.146823 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b562629-5eea-4b5c-8ed3-9d29dbff6c45 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:58.147032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b562629-5eea-4b5c-8ed3-9d29dbff6c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:58.165617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3b562629-5eea-4b5c-8ed3-9d29dbff6c45 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:52:58.166937 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3b562629-5eea-4b5c-8ed3-9d29dbff6c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:52:58.237379 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 579/1155] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:52:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:52:58.262267 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03bd5d-6e02-463e-ab5d-7624d2b34237 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:58.359756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:58.364453 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03bd5d-6e02-463e-ab5d-7624d2b34237 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:58.398949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8168a4d-abb8-4e0f-9ef8-2af08d1fdfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:58.399812 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8168a4d-abb8-4e0f-9ef8-2af08d1fdfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:58.400446 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8168a4d-abb8-4e0f-9ef8-2af08d1fdfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:58.400733 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8168a4d-abb8-4e0f-9ef8-2af08d1fdfe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:58.411408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8168a4d-abb8-4e0f-9ef8-2af08d1fdfe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-f8168a4d-abb8-4e0f-9ef8-2af08d1fdfe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:52:58.419024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:52:58.420157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:52:58.421162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 577/1156] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:52:58.426390 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b562629-5eea-4b5c-8ed3-9d29dbff6c45 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:58.493957 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:58.523953 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b562629-5eea-4b5c-8ed3-9d29dbff6c45 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:58.557758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6eb20-0d8a-4653-bd9e-1da3046723a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:58.558414 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6eb20-0d8a-4653-bd9e-1da3046723a1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:58.558701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6eb20-0d8a-4653-bd9e-1da3046723a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:58.558980 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6eb20-0d8a-4653-bd9e-1da3046723a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:58.559852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6eb20-0d8a-4653-bd9e-1da3046723a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:58.560167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6eb20-0d8a-4653-bd9e-1da3046723a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:58.561573 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6eb20-0d8a-4653-bd9e-1da3046723a1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:58.569356 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 580/1157] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:58.575940 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8168a4d-abb8-4e0f-9ef8-2af08d1fdfe4 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:58.674690 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8168a4d-abb8-4e0f-9ef8-2af08d1fdfe4 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:58.707826 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf537a-19db-42a9-b7b6-b25d9ad5c883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:58.708707 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf537a-19db-42a9-b7b6-b25d9ad5c883 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:58.708993 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf537a-19db-42a9-b7b6-b25d9ad5c883 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:58.709271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf537a-19db-42a9-b7b6-b25d9ad5c883 None admin] PATH_INFO: `/v3/users/f50f5a12f5054c9e9fc2d3d6026842c4` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:58.710452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf537a-19db-42a9-b7b6-b25d9ad5c883 None admin] RBAC: Authorizing `identity:get_user(user_id=f50f5a12f5054c9e9fc2d3d6026842c4)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:58.711882 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf537a-19db-42a9-b7b6-b25d9ad5c883 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:58.713543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 578/1158] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:52:58 2022] GET /identity/v3/users/f50f5a12f5054c9e9fc2d3d6026842c4 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:58.719377 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6eb20-0d8a-4653-bd9e-1da3046723a1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:58.817534 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6eb20-0d8a-4653-bd9e-1da3046723a1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:58.845648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:58.850487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dd232-67ba-4e7e-9a83-1160adf9a7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:58.851128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dd232-67ba-4e7e-9a83-1160adf9a7aa None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:58.851421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dd232-67ba-4e7e-9a83-1160adf9a7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:58.851703 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dd232-67ba-4e7e-9a83-1160adf9a7aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:58.917562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dd232-67ba-4e7e-9a83-1160adf9a7aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:58.919045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dd232-67ba-4e7e-9a83-1160adf9a7aa None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:59.019503 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 581/1159] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:52:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:52:59.025328 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf537a-19db-42a9-b7b6-b25d9ad5c883 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:59.052824 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:59.129587 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf537a-19db-42a9-b7b6-b25d9ad5c883 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:59.163551 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612e2ed-daa2-404f-891c-565648f2ff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:59.164437 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612e2ed-daa2-404f-891c-565648f2ff2e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:59.164724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612e2ed-daa2-404f-891c-565648f2ff2e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:59.165003 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612e2ed-daa2-404f-891c-565648f2ff2e None admin] PATH_INFO: `/v3/projects/83f4130a59674569965f400f1349f55c` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:59.166178 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612e2ed-daa2-404f-891c-565648f2ff2e None admin] RBAC: Authorizing `identity:get_project(project_id=83f4130a59674569965f400f1349f55c)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:59.167628 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612e2ed-daa2-404f-891c-565648f2ff2e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:59.169946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 579/1160] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:52:59 2022] GET /identity/v3/projects/83f4130a59674569965f400f1349f55c => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:52:59.176086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dd232-67ba-4e7e-9a83-1160adf9a7aa None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:52:59.210519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:52:59.277750 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dd232-67ba-4e7e-9a83-1160adf9a7aa None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:52:59.314241 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb24e87-7497-4aa3-96ab-beae1a4c8f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:52:59.314895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb24e87-7497-4aa3-96ab-beae1a4c8f78 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:52:59.315190 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb24e87-7497-4aa3-96ab-beae1a4c8f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:52:59.315465 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb24e87-7497-4aa3-96ab-beae1a4c8f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:52:59.316328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb24e87-7497-4aa3-96ab-beae1a4c8f78 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=83f4130a59674569965f400f1349f55c, user.id=f50f5a12f5054c9e9fc2d3d6026842c4) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:52:59.316644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb24e87-7497-4aa3-96ab-beae1a4c8f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:52:59.318080 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb24e87-7497-4aa3-96ab-beae1a4c8f78 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:52:59.323953 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 582/1161] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:52:59 2022] GET /identity/v3/role_assignments?user.id=f50f5a12f5054c9e9fc2d3d6026842c4&scope.project.id=83f4130a59674569965f400f1349f55c&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:01.566549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9001c33-045e-4a6c-877e-f1732fb5dedc None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:01.567136 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9001c33-045e-4a6c-877e-f1732fb5dedc None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:01.567136 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9001c33-045e-4a6c-877e-f1732fb5dedc None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:01.568496 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 580/1162] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:53:01.573655 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41a254-27de-4b17-a6ce-5840a752a08b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:01.574116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41a254-27de-4b17-a6ce-5840a752a08b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:01.574224 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41a254-27de-4b17-a6ce-5840a752a08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:01.591756 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ac41a254-27de-4b17-a6ce-5840a752a08b None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:01.592969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ac41a254-27de-4b17-a6ce-5840a752a08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:01.658243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 583/1163] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:53:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:53:01.682890 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9001c33-045e-4a6c-877e-f1732fb5dedc None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:01.775961 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:01.783078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9001c33-045e-4a6c-877e-f1732fb5dedc None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:01.816788 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca185a-eaec-4c97-a9d1-1f10bb9e10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:01.817432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca185a-eaec-4c97-a9d1-1f10bb9e10c3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:01.817717 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca185a-eaec-4c97-a9d1-1f10bb9e10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:01.817991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca185a-eaec-4c97-a9d1-1f10bb9e10c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:01.885099 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca185a-eaec-4c97-a9d1-1f10bb9e10c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:01.886534 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca185a-eaec-4c97-a9d1-1f10bb9e10c3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:01.975835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:01.983803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 581/1164] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:53:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:53:01.989906 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41a254-27de-4b17-a6ce-5840a752a08b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:02.016963 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:02.088680 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41a254-27de-4b17-a6ce-5840a752a08b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:02.122708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec2ed7-b111-4f56-8599-aa2bf4f2e816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:02.123503 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec2ed7-b111-4f56-8599-aa2bf4f2e816 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:02.123788 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec2ed7-b111-4f56-8599-aa2bf4f2e816 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:02.124068 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec2ed7-b111-4f56-8599-aa2bf4f2e816 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:02.130888 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec2ed7-b111-4f56-8599-aa2bf4f2e816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:02.132371 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec2ed7-b111-4f56-8599-aa2bf4f2e816 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-aeec2ed7-b111-4f56-8599-aa2bf4f2e816 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:53:02.146350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:53:02.147617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:53:02.148895 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 584/1165] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:53:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:53:02.153759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca185a-eaec-4c97-a9d1-1f10bb9e10c3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:02.259270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca185a-eaec-4c97-a9d1-1f10bb9e10c3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:02.293013 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de9dc7-754b-4ce1-aa02-6c19060f23bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:02.294200 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de9dc7-754b-4ce1-aa02-6c19060f23bb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:02.294523 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de9dc7-754b-4ce1-aa02-6c19060f23bb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:02.294766 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de9dc7-754b-4ce1-aa02-6c19060f23bb None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:02.295481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de9dc7-754b-4ce1-aa02-6c19060f23bb None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:53:02.295791 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de9dc7-754b-4ce1-aa02-6c19060f23bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:02.297180 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de9dc7-754b-4ce1-aa02-6c19060f23bb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:02.304248 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 582/1166] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 14:53:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:04.413457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7f437-90a6-413f-a747-9ced2b27b3c8 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:04.414027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7f437-90a6-413f-a747-9ced2b27b3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:04.414027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7f437-90a6-413f-a747-9ced2b27b3c8 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:04.415373 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 585/1167] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:53:04.420682 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211216-0ed1-43a7-a652-58ee0f140b4e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:04.421127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211216-0ed1-43a7-a652-58ee0f140b4e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:04.421315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211216-0ed1-43a7-a652-58ee0f140b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:04.438217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b7211216-0ed1-43a7-a652-58ee0f140b4e None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:04.439403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b7211216-0ed1-43a7-a652-58ee0f140b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:04.492585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:04.509323 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 583/1168] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:53:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:53:04.533627 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7f437-90a6-413f-a747-9ced2b27b3c8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:04.620922 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:04.638214 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7f437-90a6-413f-a747-9ced2b27b3c8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:04.670422 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c8c4b-a781-4e15-a90f-a3060d7e1067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:04.671006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c8c4b-a781-4e15-a90f-a3060d7e1067 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:04.671292 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c8c4b-a781-4e15-a90f-a3060d7e1067 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:04.671572 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c8c4b-a781-4e15-a90f-a3060d7e1067 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:04.672335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c8c4b-a781-4e15-a90f-a3060d7e1067 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:04.673706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c8c4b-a781-4e15-a90f-a3060d7e1067 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:04.815905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 586/1169] 10.222.0.88 () {60 vars in 1138 bytes} [Thu Jun 16 14:53:04 2022] POST /identity/v3/domains => generated 267 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:53:06.985287 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca463-9c7b-43c2-8d6e-3d6b3c3306ac None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:06.985853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca463-9c7b-43c2-8d6e-3d6b3c3306ac None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:06.985853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca463-9c7b-43c2-8d6e-3d6b3c3306ac None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:06.987226 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 584/1170] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:53:06.992570 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a466924-5c7c-4f97-b6d8-0d9bf3631506 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:06.993032 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a466924-5c7c-4f97-b6d8-0d9bf3631506 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:06.993129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a466924-5c7c-4f97-b6d8-0d9bf3631506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:07.009563 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-4a466924-5c7c-4f97-b6d8-0d9bf3631506 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:07.010744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-4a466924-5c7c-4f97-b6d8-0d9bf3631506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:07.077535 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 587/1171] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:53:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:53:07.101097 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca463-9c7b-43c2-8d6e-3d6b3c3306ac None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:07.202896 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca463-9c7b-43c2-8d6e-3d6b3c3306ac None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:07.236571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6721c-6d3c-4332-aac8-0616490ea884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:07.237350 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6721c-6d3c-4332-aac8-0616490ea884 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:07.237634 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6721c-6d3c-4332-aac8-0616490ea884 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:07.237908 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6721c-6d3c-4332-aac8-0616490ea884 None admin] PATH_INFO: `/v3/domains/bb0e7f5dbe7b45c699217396f0bb020d` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:07.248953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6721c-6d3c-4332-aac8-0616490ea884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bb0e7f5dbe7b45c699217396f0bb020d)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:07.250386 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6721c-6d3c-4332-aac8-0616490ea884 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:07.251994 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 585/1172] 10.222.0.88 () {56 vars in 1183 bytes} [Thu Jun 16 14:53:07 2022] GET /identity/v3/domains/bb0e7f5dbe7b45c699217396f0bb020d => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:07.258243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a466924-5c7c-4f97-b6d8-0d9bf3631506 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:07.333270 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:07.355103 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a466924-5c7c-4f97-b6d8-0d9bf3631506 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:07.390501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb25bf-1deb-4e95-a468-2b24cc455bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:07.391150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb25bf-1deb-4e95-a468-2b24cc455bac None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:07.391432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb25bf-1deb-4e95-a468-2b24cc455bac None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:07.391708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb25bf-1deb-4e95-a468-2b24cc455bac None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:07.392439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb25bf-1deb-4e95-a468-2b24cc455bac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:07.393867 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb25bf-1deb-4e95-a468-2b24cc455bac None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:07.506997 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 588/1173] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:53:07 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:53:09.673818 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1681a5f-5338-4f85-9d1b-62d32785bcc7 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:09.674352 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1681a5f-5338-4f85-9d1b-62d32785bcc7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:09.674352 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1681a5f-5338-4f85-9d1b-62d32785bcc7 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:09.675737 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 586/1174] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:53:09.681083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0190d7-2877-43ac-97a6-472e5ef5ed50 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:09.681552 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0190d7-2877-43ac-97a6-472e5ef5ed50 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:09.681656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0190d7-2877-43ac-97a6-472e5ef5ed50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:09.698607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fa0190d7-2877-43ac-97a6-472e5ef5ed50 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:09.700091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fa0190d7-2877-43ac-97a6-472e5ef5ed50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:09.766794 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 589/1175] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:53:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:53:09.791156 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1681a5f-5338-4f85-9d1b-62d32785bcc7 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:09.845815 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:09.898251 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1681a5f-5338-4f85-9d1b-62d32785bcc7 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:09.935369 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a937042-44bf-446d-ab60-4cc798c9f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:09.936182 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a937042-44bf-446d-ab60-4cc798c9f00b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:09.936498 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a937042-44bf-446d-ab60-4cc798c9f00b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:09.936774 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a937042-44bf-446d-ab60-4cc798c9f00b None admin] PATH_INFO: `/v3/domains/bb0e7f5dbe7b45c699217396f0bb020d` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:09.937938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a937042-44bf-446d-ab60-4cc798c9f00b None admin] RBAC: Authorizing `identity:get_domain(domain_id=bb0e7f5dbe7b45c699217396f0bb020d)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:09.939364 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a937042-44bf-446d-ab60-4cc798c9f00b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:09.941039 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 587/1176] 10.222.0.88 () {56 vars in 1183 bytes} [Thu Jun 16 14:53:09 2022] GET /identity/v3/domains/bb0e7f5dbe7b45c699217396f0bb020d => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:09.946979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0190d7-2877-43ac-97a6-472e5ef5ed50 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:10.003497 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:10.043118 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0190d7-2877-43ac-97a6-472e5ef5ed50 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:10.077027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497aa40-e618-467d-bedb-1571c118a6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:10.077693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497aa40-e618-467d-bedb-1571c118a6dc None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.077978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497aa40-e618-467d-bedb-1571c118a6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.078258 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497aa40-e618-467d-bedb-1571c118a6dc None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.078961 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497aa40-e618-467d-bedb-1571c118a6dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:10.080419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497aa40-e618-467d-bedb-1571c118a6dc None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:10.255117 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 590/1177] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:53:09 2022] POST /identity/v3/users => generated 311 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:53:12.451229 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28be7ca-4ba8-4ef0-8363-27cc906ee560 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:12.451795 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28be7ca-4ba8-4ef0-8363-27cc906ee560 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:12.451795 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28be7ca-4ba8-4ef0-8363-27cc906ee560 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:12.453196 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 588/1178] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:53:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:53:12.458343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5eb4-b8ca-4b9c-a3ed-5920911affa4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:12.458766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5eb4-b8ca-4b9c-a3ed-5920911affa4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:12.458912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5eb4-b8ca-4b9c-a3ed-5920911affa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:12.475660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1a0a5eb4-b8ca-4b9c-a3ed-5920911affa4 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:12.476875 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1a0a5eb4-b8ca-4b9c-a3ed-5920911affa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:12.544877 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 591/1179] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:53:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:53:12.570281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28be7ca-4ba8-4ef0-8363-27cc906ee560 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:12.670284 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28be7ca-4ba8-4ef0-8363-27cc906ee560 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:12.686945 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:12.703913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b2132-2456-4c8e-b493-248514f8a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:12.704649 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b2132-2456-4c8e-b493-248514f8a095 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:12.704935 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b2132-2456-4c8e-b493-248514f8a095 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:12.705206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b2132-2456-4c8e-b493-248514f8a095 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:12.714978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b2132-2456-4c8e-b493-248514f8a095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2b4b2132-2456-4c8e-b493-248514f8a095 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:53:12.722974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:53:12.724129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:53:12.725316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 589/1180] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:53:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:53:12.731203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a5eb4-b8ca-4b9c-a3ed-5920911affa4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:12.776620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:12.829511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a5eb4-b8ca-4b9c-a3ed-5920911affa4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:12.862824 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6211cb-4a55-4ffd-9b9b-6f1131d49bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:12.863477 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6211cb-4a55-4ffd-9b9b-6f1131d49bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:12.863766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6211cb-4a55-4ffd-9b9b-6f1131d49bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:12.864050 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6211cb-4a55-4ffd-9b9b-6f1131d49bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:12.864827 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6211cb-4a55-4ffd-9b9b-6f1131d49bc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:53:12.865140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6211cb-4a55-4ffd-9b9b-6f1131d49bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:12.866552 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6211cb-4a55-4ffd-9b9b-6f1131d49bc6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:12.875332 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 592/1181] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:53:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:12.881463 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b2132-2456-4c8e-b493-248514f8a095 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:12.979719 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b2132-2456-4c8e-b493-248514f8a095 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:13.013484 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e744a67-da02-4817-82ce-6a7ceb340d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:13.014347 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e744a67-da02-4817-82ce-6a7ceb340d4d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:13.014636 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e744a67-da02-4817-82ce-6a7ceb340d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:13.014917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e744a67-da02-4817-82ce-6a7ceb340d4d None admin] PATH_INFO: `/v3/users/9c3d72e3dd1f4665afc4dc58a716422f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:13.029205 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e744a67-da02-4817-82ce-6a7ceb340d4d None admin] RBAC: Authorizing `identity:get_user(user_id=9c3d72e3dd1f4665afc4dc58a716422f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:13.030651 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e744a67-da02-4817-82ce-6a7ceb340d4d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:13.032318 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 590/1182] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:53:12 2022] GET /identity/v3/users/9c3d72e3dd1f4665afc4dc58a716422f => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:13.038070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6211cb-4a55-4ffd-9b9b-6f1131d49bc6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:13.138362 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6211cb-4a55-4ffd-9b9b-6f1131d49bc6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:13.144973 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:13.171693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0420a-03ba-4eb4-a4c9-9f94d167dea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:13.172356 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0420a-03ba-4eb4-a4c9-9f94d167dea1 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:13.172644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0420a-03ba-4eb4-a4c9-9f94d167dea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:13.172926 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0420a-03ba-4eb4-a4c9-9f94d167dea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:13.237861 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0420a-03ba-4eb4-a4c9-9f94d167dea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:13.239254 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0420a-03ba-4eb4-a4c9-9f94d167dea1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:13.336816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 593/1183] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:53:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:53:13.343379 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e744a67-da02-4817-82ce-6a7ceb340d4d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:13.376712 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:13.441978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e744a67-da02-4817-82ce-6a7ceb340d4d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:13.475368 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d22587-4793-411c-8ff2-43d49c6fd3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:13.476211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d22587-4793-411c-8ff2-43d49c6fd3bb None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:13.476589 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d22587-4793-411c-8ff2-43d49c6fd3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:13.476862 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d22587-4793-411c-8ff2-43d49c6fd3bb None admin] PATH_INFO: `/v3/projects/0f5493ec697c4606a0b1a75d2ebf5257` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:13.478016 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d22587-4793-411c-8ff2-43d49c6fd3bb None admin] RBAC: Authorizing `identity:get_project(project_id=0f5493ec697c4606a0b1a75d2ebf5257)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:13.479429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d22587-4793-411c-8ff2-43d49c6fd3bb None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:13.481269 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 591/1184] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:53:13 2022] GET /identity/v3/projects/0f5493ec697c4606a0b1a75d2ebf5257 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:13.487309 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0420a-03ba-4eb4-a4c9-9f94d167dea1 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:13.495478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:13.585010 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0420a-03ba-4eb4-a4c9-9f94d167dea1 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:13.618219 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11043c65-3d19-40e2-b675-0b0b28060fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:13.618865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043c65-3d19-40e2-b675-0b0b28060fba None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:13.619154 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043c65-3d19-40e2-b675-0b0b28060fba None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:13.619435 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043c65-3d19-40e2-b675-0b0b28060fba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:13.620279 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043c65-3d19-40e2-b675-0b0b28060fba None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=0f5493ec697c4606a0b1a75d2ebf5257, user.id=9c3d72e3dd1f4665afc4dc58a716422f) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:53:13.620599 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043c65-3d19-40e2-b675-0b0b28060fba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:13.622576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11043c65-3d19-40e2-b675-0b0b28060fba None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:13.627912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 594/1185] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:53:13 2022] GET /identity/v3/role_assignments?user.id=9c3d72e3dd1f4665afc4dc58a716422f&scope.project.id=0f5493ec697c4606a0b1a75d2ebf5257&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:15.725420 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b61f38-4a2f-47b2-a6fb-d3cb9c56d2c1 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:15.725982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b61f38-4a2f-47b2-a6fb-d3cb9c56d2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:15.725982 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b61f38-4a2f-47b2-a6fb-d3cb9c56d2c1 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:15.727343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 592/1186] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:53:15.733052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb6664-ee7b-4802-9287-0f3433cdabd1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:15.733515 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb6664-ee7b-4802-9287-0f3433cdabd1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:15.733621 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb6664-ee7b-4802-9287-0f3433cdabd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:15.750729 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-76eb6664-ee7b-4802-9287-0f3433cdabd1 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:15.751909 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-76eb6664-ee7b-4802-9287-0f3433cdabd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:15.813649 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:15.818170 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 595/1187] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:53:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:53:15.840600 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b61f38-4a2f-47b2-a6fb-d3cb9c56d2c1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:15.941210 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b61f38-4a2f-47b2-a6fb-d3cb9c56d2c1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:15.942727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:15.974518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3e032-7695-4e3a-aae8-b09354e65bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:15.975300 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3e032-7695-4e3a-aae8-b09354e65bab None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:15.975588 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3e032-7695-4e3a-aae8-b09354e65bab None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:15.975869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3e032-7695-4e3a-aae8-b09354e65bab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:15.986739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3e032-7695-4e3a-aae8-b09354e65bab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b2d3e032-7695-4e3a-aae8-b09354e65bab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:53:15.994914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:53:15.996460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:53:15.997407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 593/1188] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:53:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:53:16.003100 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb6664-ee7b-4802-9287-0f3433cdabd1 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:16.100325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb6664-ee7b-4802-9287-0f3433cdabd1 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:16.133891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38839c29-87bc-4f48-9e40-ab6b76ee0027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:16.134554 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38839c29-87bc-4f48-9e40-ab6b76ee0027 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:16.134845 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38839c29-87bc-4f48-9e40-ab6b76ee0027 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:16.135127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38839c29-87bc-4f48-9e40-ab6b76ee0027 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:16.135873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38839c29-87bc-4f48-9e40-ab6b76ee0027 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:53:16.136184 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38839c29-87bc-4f48-9e40-ab6b76ee0027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:16.137692 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38839c29-87bc-4f48-9e40-ab6b76ee0027 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:16.145615 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 596/1189] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:53:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:16.151544 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3e032-7695-4e3a-aae8-b09354e65bab None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:16.251934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3e032-7695-4e3a-aae8-b09354e65bab None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:16.285287 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0bcd0-0c78-42aa-9846-8722e9d56d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:16.286137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0bcd0-0c78-42aa-9846-8722e9d56d5e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:16.286421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0bcd0-0c78-42aa-9846-8722e9d56d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:16.286698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0bcd0-0c78-42aa-9846-8722e9d56d5e None admin] PATH_INFO: `/v3/users/9c3d72e3dd1f4665afc4dc58a716422f` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:16.287873 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0bcd0-0c78-42aa-9846-8722e9d56d5e None admin] RBAC: Authorizing `identity:get_user(user_id=9c3d72e3dd1f4665afc4dc58a716422f)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:16.289339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0bcd0-0c78-42aa-9846-8722e9d56d5e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:16.290949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 594/1190] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:53:16 2022] GET /identity/v3/users/9c3d72e3dd1f4665afc4dc58a716422f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:16.296829 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38839c29-87bc-4f48-9e40-ab6b76ee0027 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:16.357753 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:16.396460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38839c29-87bc-4f48-9e40-ab6b76ee0027 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:16.430787 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de953ded-314f-4670-98be-d5919edb7831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:16.431626 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de953ded-314f-4670-98be-d5919edb7831 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:16.431909 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de953ded-314f-4670-98be-d5919edb7831 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:16.432182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de953ded-314f-4670-98be-d5919edb7831 None admin] PATH_INFO: `/v3/projects/0f5493ec697c4606a0b1a75d2ebf5257` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:16.433555 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de953ded-314f-4670-98be-d5919edb7831 None admin] RBAC: Authorizing `identity:get_project(project_id=0f5493ec697c4606a0b1a75d2ebf5257)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:16.435357 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de953ded-314f-4670-98be-d5919edb7831 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:16.437077 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 597/1191] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:53:16 2022] GET /identity/v3/projects/0f5493ec697c4606a0b1a75d2ebf5257 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:16.443043 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0bcd0-0c78-42aa-9846-8722e9d56d5e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:16.474316 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:16.544291 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0bcd0-0c78-42aa-9846-8722e9d56d5e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:16.577355 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342a344-cf6f-4b2d-8345-1053d38392af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:16.578306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342a344-cf6f-4b2d-8345-1053d38392af None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:16.578594 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342a344-cf6f-4b2d-8345-1053d38392af None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:16.578876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342a344-cf6f-4b2d-8345-1053d38392af None admin] PATH_INFO: `/v3/projects/0f5493ec697c4606a0b1a75d2ebf5257/users/9c3d72e3dd1f4665afc4dc58a716422f/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:16.580907 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342a344-cf6f-4b2d-8345-1053d38392af None admin] RBAC: Authorizing `identity:create_grant(project_id=0f5493ec697c4606a0b1a75d2ebf5257, user_id=9c3d72e3dd1f4665afc4dc58a716422f, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:16.582338 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342a344-cf6f-4b2d-8345-1053d38392af None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:16.656500 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 595/1192] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:53:16 2022] PUT /identity/v3/projects/0f5493ec697c4606a0b1a75d2ebf5257/users/9c3d72e3dd1f4665afc4dc58a716422f/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:53:18.768480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8969c4-d449-4fda-8d70-f39e45780ec3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:18.769013 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8969c4-d449-4fda-8d70-f39e45780ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:18.769013 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8969c4-d449-4fda-8d70-f39e45780ec3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:18.770396 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 598/1193] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:53:18.775874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48fbc8-fed4-4d25-9835-1c9e609fb784 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:18.776325 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48fbc8-fed4-4d25-9835-1c9e609fb784 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:18.776470 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48fbc8-fed4-4d25-9835-1c9e609fb784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:18.794259 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9e48fbc8-fed4-4d25-9835-1c9e609fb784 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:18.795455 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9e48fbc8-fed4-4d25-9835-1c9e609fb784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:18.864680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 596/1194] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:53:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:53:18.890098 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8969c4-d449-4fda-8d70-f39e45780ec3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:18.996821 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8969c4-d449-4fda-8d70-f39e45780ec3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:19.026187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:19.031984 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af37591-c8a5-40f4-a9b1-32da3c51fae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:19.032814 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af37591-c8a5-40f4-a9b1-32da3c51fae0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:19.033103 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af37591-c8a5-40f4-a9b1-32da3c51fae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:19.033386 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af37591-c8a5-40f4-a9b1-32da3c51fae0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:19.044313 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af37591-c8a5-40f4-a9b1-32da3c51fae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1af37591-c8a5-40f4-a9b1-32da3c51fae0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:53:19.051816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:53:19.053087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:53:19.053959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 599/1195] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:53:19.063253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48fbc8-fed4-4d25-9835-1c9e609fb784 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:19.085414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:19.172339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48fbc8-fed4-4d25-9835-1c9e609fb784 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:19.209463 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76dca-93b0-4650-89e6-9c806f302ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:19.210127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76dca-93b0-4650-89e6-9c806f302ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:19.210414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76dca-93b0-4650-89e6-9c806f302ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:19.210692 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76dca-93b0-4650-89e6-9c806f302ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:19.211480 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a76dca-93b0-4650-89e6-9c806f302ce3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:53:19.211787 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a76dca-93b0-4650-89e6-9c806f302ce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:19.213197 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a76dca-93b0-4650-89e6-9c806f302ce3 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:19.221794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 597/1196] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:53:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:19.227999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af37591-c8a5-40f4-a9b1-32da3c51fae0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:19.332350 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af37591-c8a5-40f4-a9b1-32da3c51fae0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:19.367660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e8f5b-de15-478e-82e9-d6b9d61b3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:19.368553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e8f5b-de15-478e-82e9-d6b9d61b3d00 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:19.368843 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e8f5b-de15-478e-82e9-d6b9d61b3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:19.369128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e8f5b-de15-478e-82e9-d6b9d61b3d00 None admin] PATH_INFO: `/v3/users/9c3d72e3dd1f4665afc4dc58a716422f` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:19.370340 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e8f5b-de15-478e-82e9-d6b9d61b3d00 None admin] RBAC: Authorizing `identity:get_user(user_id=9c3d72e3dd1f4665afc4dc58a716422f)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:19.372435 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e8f5b-de15-478e-82e9-d6b9d61b3d00 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:19.374087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 600/1197] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 14:53:19 2022] GET /identity/v3/users/9c3d72e3dd1f4665afc4dc58a716422f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:19.380174 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76dca-93b0-4650-89e6-9c806f302ce3 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:19.463746 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:19.485886 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76dca-93b0-4650-89e6-9c806f302ce3 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:19.525923 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a135ba0-e3e9-4ff8-a42e-c381714d3a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:19.526582 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a135ba0-e3e9-4ff8-a42e-c381714d3a9b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:19.526911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a135ba0-e3e9-4ff8-a42e-c381714d3a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:19.527242 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a135ba0-e3e9-4ff8-a42e-c381714d3a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:19.601570 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a135ba0-e3e9-4ff8-a42e-c381714d3a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:19.602942 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a135ba0-e3e9-4ff8-a42e-c381714d3a9b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:19.686820 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:19.705552 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 598/1198] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:53:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:53:19.711616 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e8f5b-de15-478e-82e9-d6b9d61b3d00 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:19.784790 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:19.827763 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e8f5b-de15-478e-82e9-d6b9d61b3d00 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:19.865184 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06721269-ed4e-495b-82a5-cf816ef1a8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:19.865927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06721269-ed4e-495b-82a5-cf816ef1a8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:19.866211 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06721269-ed4e-495b-82a5-cf816ef1a8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:19.866494 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06721269-ed4e-495b-82a5-cf816ef1a8c2 None admin] PATH_INFO: `/v3/projects/0f5493ec697c4606a0b1a75d2ebf5257` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:19.867673 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06721269-ed4e-495b-82a5-cf816ef1a8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=0f5493ec697c4606a0b1a75d2ebf5257)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:19.869140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06721269-ed4e-495b-82a5-cf816ef1a8c2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:19.871431 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 601/1199] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 14:53:19 2022] GET /identity/v3/projects/0f5493ec697c4606a0b1a75d2ebf5257 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:19.877497 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a135ba0-e3e9-4ff8-a42e-c381714d3a9b None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:19.983664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a135ba0-e3e9-4ff8-a42e-c381714d3a9b None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:20.021987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdfaf0-9b0b-46ee-9577-d45f65bd5e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:20.022641 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdfaf0-9b0b-46ee-9577-d45f65bd5e6b None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:20.022968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdfaf0-9b0b-46ee-9577-d45f65bd5e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:20.023249 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdfaf0-9b0b-46ee-9577-d45f65bd5e6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:20.024079 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdfaf0-9b0b-46ee-9577-d45f65bd5e6b None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=0f5493ec697c4606a0b1a75d2ebf5257, user.id=9c3d72e3dd1f4665afc4dc58a716422f) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:53:20.024424 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdfaf0-9b0b-46ee-9577-d45f65bd5e6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:20.025866 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdfaf0-9b0b-46ee-9577-d45f65bd5e6b None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:20.032329 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 599/1200] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:53:19 2022] GET /identity/v3/role_assignments?user.id=9c3d72e3dd1f4665afc4dc58a716422f&scope.project.id=0f5493ec697c4606a0b1a75d2ebf5257&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:29.860096 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=37 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:30.029596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=77 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:42.058637 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7845781-6961-40d1-9b5d-23d242bd0fa4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:42.059545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7845781-6961-40d1-9b5d-23d242bd0fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:42.059545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7845781-6961-40d1-9b5d-23d242bd0fa4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:42.061178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 602/1201] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:54:42.067336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7aaaa-1c1e-4c05-8e0e-be4245b8ee2b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:42.067893 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7aaaa-1c1e-4c05-8e0e-be4245b8ee2b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:42.068022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7aaaa-1c1e-4c05-8e0e-be4245b8ee2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:42.085967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b9b7aaaa-1c1e-4c05-8e0e-be4245b8ee2b None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:54:42.087371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b9b7aaaa-1c1e-4c05-8e0e-be4245b8ee2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:54:42.163037 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 600/1202] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:54:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:54:42.188718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7845781-6961-40d1-9b5d-23d242bd0fa4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:42.305926 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7845781-6961-40d1-9b5d-23d242bd0fa4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:42.341747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa354f09-55fc-44e3-9cd9-2ad338ea77b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:42.342446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa354f09-55fc-44e3-9cd9-2ad338ea77b4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:42.342732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa354f09-55fc-44e3-9cd9-2ad338ea77b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:42.343010 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa354f09-55fc-44e3-9cd9-2ad338ea77b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:42.354129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa354f09-55fc-44e3-9cd9-2ad338ea77b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:42.355593 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa354f09-55fc-44e3-9cd9-2ad338ea77b4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-aa354f09-55fc-44e3-9cd9-2ad338ea77b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:54:42.375099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:42.376469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:42.377472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:54:42.377472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:54:42.377472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:42.377472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:54:42.377472 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 603/1203] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:54:42.383591 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7aaaa-1c1e-4c05-8e0e-be4245b8ee2b None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:42.490404 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7aaaa-1c1e-4c05-8e0e-be4245b8ee2b None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:42.525607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:42.531532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d6969-1d50-46fc-98f8-d75f2e421084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:42.532234 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6969-1d50-46fc-98f8-d75f2e421084 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:42.532536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6969-1d50-46fc-98f8-d75f2e421084 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:42.532822 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6969-1d50-46fc-98f8-d75f2e421084 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:42.533559 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d6969-1d50-46fc-98f8-d75f2e421084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:42.533872 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d6969-1d50-46fc-98f8-d75f2e421084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:42.535251 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d6969-1d50-46fc-98f8-d75f2e421084 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:42.547474 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 601/1204] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:42.553710 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa354f09-55fc-44e3-9cd9-2ad338ea77b4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:42.612029 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:42.661787 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa354f09-55fc-44e3-9cd9-2ad338ea77b4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:42.698534 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24945154-803a-4a27-b7a7-004230f330ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:42.699213 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24945154-803a-4a27-b7a7-004230f330ad None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:42.699502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24945154-803a-4a27-b7a7-004230f330ad None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:42.699782 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24945154-803a-4a27-b7a7-004230f330ad None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:42.700543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24945154-803a-4a27-b7a7-004230f330ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:42.701980 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24945154-803a-4a27-b7a7-004230f330ad None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:42.788658 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 604/1205] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 14:54:42 2022] POST /identity/v3/users => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:54:45.064661 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37e2f7-0d1f-4099-b55b-9fd2ab55bbb3 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:45.065224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37e2f7-0d1f-4099-b55b-9fd2ab55bbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:45.065224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37e2f7-0d1f-4099-b55b-9fd2ab55bbb3 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:45.066659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 602/1206] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:54:45.073137 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12e0fb-c134-4422-9f2a-b510e5f58ff1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:45.073612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12e0fb-c134-4422-9f2a-b510e5f58ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:45.073718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12e0fb-c134-4422-9f2a-b510e5f58ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:45.093015 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5f12e0fb-c134-4422-9f2a-b510e5f58ff1 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:54:45.094216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5f12e0fb-c134-4422-9f2a-b510e5f58ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:54:45.162523 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 605/1207] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:54:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:54:45.186984 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37e2f7-0d1f-4099-b55b-9fd2ab55bbb3 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:45.294985 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37e2f7-0d1f-4099-b55b-9fd2ab55bbb3 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:45.330290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71236c24-6251-47e4-bb03-5971436651f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:45.331012 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71236c24-6251-47e4-bb03-5971436651f2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:45.331298 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71236c24-6251-47e4-bb03-5971436651f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:45.331581 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71236c24-6251-47e4-bb03-5971436651f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:45.342803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71236c24-6251-47e4-bb03-5971436651f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-71236c24-6251-47e4-bb03-5971436651f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:54:45.351224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:45.352421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:45.353632 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 603/1208] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:54:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:45.358978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12e0fb-c134-4422-9f2a-b510e5f58ff1 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:45.401946 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:45.461326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12e0fb-c134-4422-9f2a-b510e5f58ff1 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:45.499092 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03980170-fdb8-4568-a609-64191de1a560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:45.499812 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03980170-fdb8-4568-a609-64191de1a560 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:45.500114 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03980170-fdb8-4568-a609-64191de1a560 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:45.500461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03980170-fdb8-4568-a609-64191de1a560 None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:45.501233 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03980170-fdb8-4568-a609-64191de1a560 None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:45.501776 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03980170-fdb8-4568-a609-64191de1a560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:45.503841 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03980170-fdb8-4568-a609-64191de1a560 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:45.513620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 606/1209] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:54:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:45.519621 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71236c24-6251-47e4-bb03-5971436651f2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:45.571927 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:45.623856 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71236c24-6251-47e4-bb03-5971436651f2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:45.658870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62904fbe-1640-4917-8028-35b020165144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:45.659660 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62904fbe-1640-4917-8028-35b020165144 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:45.660524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62904fbe-1640-4917-8028-35b020165144 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:45.660813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62904fbe-1640-4917-8028-35b020165144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:45.672132 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62904fbe-1640-4917-8028-35b020165144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:45.673678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62904fbe-1640-4917-8028-35b020165144 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-62904fbe-1640-4917-8028-35b020165144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:54:45.690968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:45.696996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:45.698175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:54:45.698175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:54:45.698175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:45.698175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:45.698175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 604/1210] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:45.698538 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03980170-fdb8-4568-a609-64191de1a560 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:45.808197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:45.811127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03980170-fdb8-4568-a609-64191de1a560 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:45.843457 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7a075-3e1e-4029-86f3-886baa64347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:45.844119 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7a075-3e1e-4029-86f3-886baa64347c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:45.844438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7a075-3e1e-4029-86f3-886baa64347c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:45.844723 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7a075-3e1e-4029-86f3-886baa64347c None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:45.845446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7a075-3e1e-4029-86f3-886baa64347c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:45.845761 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7a075-3e1e-4029-86f3-886baa64347c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:45.847142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7a075-3e1e-4029-86f3-886baa64347c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:45.859871 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 607/1211] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:45.865986 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62904fbe-1640-4917-8028-35b020165144 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:45.969317 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62904fbe-1640-4917-8028-35b020165144 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:45.970805 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:46.005624 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93886d8d-c6da-430b-833e-32ad913f6a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:46.013981 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93886d8d-c6da-430b-833e-32ad913f6a4f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:46.013981 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93886d8d-c6da-430b-833e-32ad913f6a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:46.013981 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93886d8d-c6da-430b-833e-32ad913f6a4f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:46.015425 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93886d8d-c6da-430b-833e-32ad913f6a4f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:46.016962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93886d8d-c6da-430b-833e-32ad913f6a4f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-93886d8d-c6da-430b-833e-32ad913f6a4f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:54:46.032083 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:46.033995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:46.036414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 605/1212] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:46.042079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7a075-3e1e-4029-86f3-886baa64347c None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:46.148759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7a075-3e1e-4029-86f3-886baa64347c None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:46.183696 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148a9e4-bb26-4ea0-9e89-e0a8858f50a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:46.184409 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148a9e4-bb26-4ea0-9e89-e0a8858f50a0 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:46.184694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148a9e4-bb26-4ea0-9e89-e0a8858f50a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:46.184978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148a9e4-bb26-4ea0-9e89-e0a8858f50a0 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:46.185854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148a9e4-bb26-4ea0-9e89-e0a8858f50a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:46.186185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148a9e4-bb26-4ea0-9e89-e0a8858f50a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:46.187678 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148a9e4-bb26-4ea0-9e89-e0a8858f50a0 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:46.205342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 608/1213] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 14:54:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:46.211840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93886d8d-c6da-430b-833e-32ad913f6a4f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:46.316778 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93886d8d-c6da-430b-833e-32ad913f6a4f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:46.350626 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48038c1a-887e-45b5-a21f-2c13ae6d716f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:46.351773 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48038c1a-887e-45b5-a21f-2c13ae6d716f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:46.352060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48038c1a-887e-45b5-a21f-2c13ae6d716f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:46.352516 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48038c1a-887e-45b5-a21f-2c13ae6d716f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:46.384824 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:46.422294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48038c1a-887e-45b5-a21f-2c13ae6d716f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:46.423747 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48038c1a-887e-45b5-a21f-2c13ae6d716f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:46.520178 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 606/1214] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:54:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:54:46.526511 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148a9e4-bb26-4ea0-9e89-e0a8858f50a0 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:46.546126 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:46.634928 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148a9e4-bb26-4ea0-9e89-e0a8858f50a0 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:46.671262 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2629a-1457-42a2-b841-631c8dcf8c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:46.672063 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2629a-1457-42a2-b841-631c8dcf8c00 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:46.672387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2629a-1457-42a2-b841-631c8dcf8c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:46.672694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2629a-1457-42a2-b841-631c8dcf8c00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:46.684027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2629a-1457-42a2-b841-631c8dcf8c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:46.685535 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2629a-1457-42a2-b841-631c8dcf8c00 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-5cb2629a-1457-42a2-b841-631c8dcf8c00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:54:46.705458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:46.706765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:46.707766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:54:46.707766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:54:46.707766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:46.707766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:54:46.707766 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 609/1215] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:46.713532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48038c1a-887e-45b5-a21f-2c13ae6d716f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:46.787976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:46.829537 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48038c1a-887e-45b5-a21f-2c13ae6d716f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:46.863520 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea11597-b436-482d-9083-b4035993dda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:46.864308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11597-b436-482d-9083-b4035993dda6 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:46.864598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11597-b436-482d-9083-b4035993dda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:46.864881 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11597-b436-482d-9083-b4035993dda6 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:46.865597 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea11597-b436-482d-9083-b4035993dda6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:46.865963 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea11597-b436-482d-9083-b4035993dda6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:46.867531 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea11597-b436-482d-9083-b4035993dda6 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:46.878900 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 607/1216] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:46.885525 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2629a-1457-42a2-b841-631c8dcf8c00 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:46.955579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:46.986917 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2629a-1457-42a2-b841-631c8dcf8c00 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:47.021002 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3252ce1-92b0-4006-aeee-9fc153c84d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:47.021891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3252ce1-92b0-4006-aeee-9fc153c84d2d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:47.022217 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3252ce1-92b0-4006-aeee-9fc153c84d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:47.022521 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3252ce1-92b0-4006-aeee-9fc153c84d2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:47.030197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3252ce1-92b0-4006-aeee-9fc153c84d2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:47.036540 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3252ce1-92b0-4006-aeee-9fc153c84d2d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-e3252ce1-92b0-4006-aeee-9fc153c84d2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:54:47.045887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:47.046979 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:47.048306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:54:47.048306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:54:47.048306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:54:47.048306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:54:47.048306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:54:47.048306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:54:47.048306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:54:47.048306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:54:47.048306 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 610/1217] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:54:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:47.053098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea11597-b436-482d-9083-b4035993dda6 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:47.156762 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea11597-b436-482d-9083-b4035993dda6 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:47.179096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:47.191121 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830cd6a-751b-464c-844d-abace3d0569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:47.201972 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830cd6a-751b-464c-844d-abace3d0569e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:47.201972 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830cd6a-751b-464c-844d-abace3d0569e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:47.201972 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830cd6a-751b-464c-844d-abace3d0569e None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:47.201972 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830cd6a-751b-464c-844d-abace3d0569e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:47.201972 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830cd6a-751b-464c-844d-abace3d0569e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:47.201972 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830cd6a-751b-464c-844d-abace3d0569e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:47.207366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 608/1218] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:54:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:54:47.214687 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3252ce1-92b0-4006-aeee-9fc153c84d2d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:47.318095 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3252ce1-92b0-4006-aeee-9fc153c84d2d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:47.346780 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:47.352855 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07eb0d-05b3-4b99-82dc-f1c752a87d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:47.353516 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07eb0d-05b3-4b99-82dc-f1c752a87d58 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:47.353807 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07eb0d-05b3-4b99-82dc-f1c752a87d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:47.354094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07eb0d-05b3-4b99-82dc-f1c752a87d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:47.354926 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07eb0d-05b3-4b99-82dc-f1c752a87d58 None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=8824338846a940f0865b616ec42cce61) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:47.355244 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07eb0d-05b3-4b99-82dc-f1c752a87d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:47.356813 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07eb0d-05b3-4b99-82dc-f1c752a87d58 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:47.362433 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 611/1219] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:54:47 2022] GET /identity/v3/role_assignments?user.id=8824338846a940f0865b616ec42cce61&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:49.516341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b2577-dd7f-4a47-b117-8535efccb00f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:49.516795 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b2577-dd7f-4a47-b117-8535efccb00f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:49.516889 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b2577-dd7f-4a47-b117-8535efccb00f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:49.518306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 609/1220] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:54:49.523400 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e939c-c17c-4cfc-ac31-32aab9b2e6c3 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:49.530091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e939c-c17c-4cfc-ac31-32aab9b2e6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:49.530091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e939c-c17c-4cfc-ac31-32aab9b2e6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:49.541372 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-3d8e939c-c17c-4cfc-ac31-32aab9b2e6c3 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:54:49.542556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3d8e939c-c17c-4cfc-ac31-32aab9b2e6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:54:49.609815 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 612/1221] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:54:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:54:49.633270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b2577-dd7f-4a47-b117-8535efccb00f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:49.738604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b2577-dd7f-4a47-b117-8535efccb00f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:49.774215 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0e854-614e-42eb-bfa9-89f35f9dc2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:49.774921 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0e854-614e-42eb-bfa9-89f35f9dc2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:49.775206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0e854-614e-42eb-bfa9-89f35f9dc2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:49.775482 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0e854-614e-42eb-bfa9-89f35f9dc2c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:49.785343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0e854-614e-42eb-bfa9-89f35f9dc2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-0db0e854-614e-42eb-bfa9-89f35f9dc2c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:54:49.794046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:49.795241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:49.796413 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 610/1222] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:54:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:49.801527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e939c-c17c-4cfc-ac31-32aab9b2e6c3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:49.911155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e939c-c17c-4cfc-ac31-32aab9b2e6c3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:49.935029 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:49.945985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cb178-45b3-497e-a31e-25d5e0119b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:49.946690 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cb178-45b3-497e-a31e-25d5e0119b9d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:49.946960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cb178-45b3-497e-a31e-25d5e0119b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:49.947242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cb178-45b3-497e-a31e-25d5e0119b9d None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:49.947985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cb178-45b3-497e-a31e-25d5e0119b9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:49.948327 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cb178-45b3-497e-a31e-25d5e0119b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:49.949699 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cb178-45b3-497e-a31e-25d5e0119b9d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:49.957621 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 613/1223] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:54:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:49.964181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0e854-614e-42eb-bfa9-89f35f9dc2c2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:50.008075 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:50.063051 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0e854-614e-42eb-bfa9-89f35f9dc2c2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:50.096021 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf7f5d-5eaa-447e-8ae2-1fe3966a5ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:50.096841 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7f5d-5eaa-447e-8ae2-1fe3966a5ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:50.097133 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7f5d-5eaa-447e-8ae2-1fe3966a5ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:50.097414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7f5d-5eaa-447e-8ae2-1fe3966a5ad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:50.108592 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf7f5d-5eaa-447e-8ae2-1fe3966a5ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:50.110110 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf7f5d-5eaa-447e-8ae2-1fe3966a5ad7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-afdf7f5d-5eaa-447e-8ae2-1fe3966a5ad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:54:50.126813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:50.127892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:50.133741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:54:50.133741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:54:50.133741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:50.133741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:50.133741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 611/1224] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:50.134792 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cb178-45b3-497e-a31e-25d5e0119b9d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:50.238073 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cb178-45b3-497e-a31e-25d5e0119b9d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:50.272081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb33328-1176-4da8-aacf-f1922e6c22f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:50.273452 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb33328-1176-4da8-aacf-f1922e6c22f6 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:50.273892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb33328-1176-4da8-aacf-f1922e6c22f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:50.273985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb33328-1176-4da8-aacf-f1922e6c22f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:50.274716 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb33328-1176-4da8-aacf-f1922e6c22f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:50.275026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb33328-1176-4da8-aacf-f1922e6c22f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:50.276437 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb33328-1176-4da8-aacf-f1922e6c22f6 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:50.288489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 614/1225] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:50.294565 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf7f5d-5eaa-447e-8ae2-1fe3966a5ad7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:50.390085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:50.396524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf7f5d-5eaa-447e-8ae2-1fe3966a5ad7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:50.430911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63284045-cfce-4eeb-849d-65bdc4f84585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:50.431710 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63284045-cfce-4eeb-849d-65bdc4f84585 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:50.431996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63284045-cfce-4eeb-849d-65bdc4f84585 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:50.432300 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63284045-cfce-4eeb-849d-65bdc4f84585 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:50.440136 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63284045-cfce-4eeb-849d-65bdc4f84585 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:50.441601 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63284045-cfce-4eeb-849d-65bdc4f84585 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-63284045-cfce-4eeb-849d-65bdc4f84585 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:54:50.457155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:50.458326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:50.459357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 612/1226] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:50.465268 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb33328-1176-4da8-aacf-f1922e6c22f6 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:50.515522 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:50.569091 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb33328-1176-4da8-aacf-f1922e6c22f6 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:50.604404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680066-c958-4ab0-b497-81d59e7cde29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:50.605076 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680066-c958-4ab0-b497-81d59e7cde29 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:50.605366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680066-c958-4ab0-b497-81d59e7cde29 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:50.605650 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32680066-c958-4ab0-b497-81d59e7cde29 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:50.606506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680066-c958-4ab0-b497-81d59e7cde29 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:50.606843 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680066-c958-4ab0-b497-81d59e7cde29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:50.608326 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32680066-c958-4ab0-b497-81d59e7cde29 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:50.626042 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 615/1227] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 14:54:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:50.632039 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63284045-cfce-4eeb-849d-65bdc4f84585 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:50.736672 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63284045-cfce-4eeb-849d-65bdc4f84585 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:50.772132 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6cf70-42b6-47ea-a907-c8911dfa372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:50.773098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6cf70-42b6-47ea-a907-c8911dfa372d None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:50.773404 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6cf70-42b6-47ea-a907-c8911dfa372d None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:50.773686 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6cf70-42b6-47ea-a907-c8911dfa372d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:50.784093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6cf70-42b6-47ea-a907-c8911dfa372d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:50.785573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6cf70-42b6-47ea-a907-c8911dfa372d None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-2ef6cf70-42b6-47ea-a907-c8911dfa372d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:54:50.804858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:50.805983 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:50.807745 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:54:50.807745 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:54:50.807745 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:50.807745 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:50.808400 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 613/1228] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:50.811806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680066-c958-4ab0-b497-81d59e7cde29 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:50.912451 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32680066-c958-4ab0-b497-81d59e7cde29 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:50.915795 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:50.947933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6208ae-35a4-496d-b6c9-911e3ea0764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:50.948639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6208ae-35a4-496d-b6c9-911e3ea0764e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:50.948930 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6208ae-35a4-496d-b6c9-911e3ea0764e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:50.949214 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6208ae-35a4-496d-b6c9-911e3ea0764e None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:50.949940 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6208ae-35a4-496d-b6c9-911e3ea0764e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:50.950254 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6208ae-35a4-496d-b6c9-911e3ea0764e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:50.951844 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6208ae-35a4-496d-b6c9-911e3ea0764e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:50.963476 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 616/1229] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:50.969217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6cf70-42b6-47ea-a907-c8911dfa372d None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:50.978831 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:51.075338 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6cf70-42b6-47ea-a907-c8911dfa372d None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:51.110543 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdfb3b-87e9-4a69-a2ad-f540e13383bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:51.111323 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdfb3b-87e9-4a69-a2ad-f540e13383bc None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:51.111604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdfb3b-87e9-4a69-a2ad-f540e13383bc None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:51.111880 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdfb3b-87e9-4a69-a2ad-f540e13383bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:51.119585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdfb3b-87e9-4a69-a2ad-f540e13383bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:51.121178 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdfb3b-87e9-4a69-a2ad-f540e13383bc None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-3ecdfb3b-87e9-4a69-a2ad-f540e13383bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:54:51.135625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:51.136810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:51.142501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:54:51.142501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:54:51.142501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:54:51.142501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:54:51.142501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:54:51.142501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:54:51.142501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:54:51.142501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:51.142501 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 614/1230] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:54:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:51.143012 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6208ae-35a4-496d-b6c9-911e3ea0764e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:51.250933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6208ae-35a4-496d-b6c9-911e3ea0764e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:51.284030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f852e1d-a192-467d-b1b3-b6e770e76ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:51.284716 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f852e1d-a192-467d-b1b3-b6e770e76ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:51.285005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f852e1d-a192-467d-b1b3-b6e770e76ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:51.285282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f852e1d-a192-467d-b1b3-b6e770e76ef9 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:51.286116 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f852e1d-a192-467d-b1b3-b6e770e76ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:51.286450 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f852e1d-a192-467d-b1b3-b6e770e76ef9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:51.287880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f852e1d-a192-467d-b1b3-b6e770e76ef9 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:51.299672 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 617/1231] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:54:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:54:51.306770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdfb3b-87e9-4a69-a2ad-f540e13383bc None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:51.374174 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:51.408437 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdfb3b-87e9-4a69-a2ad-f540e13383bc None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:51.441643 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07c6f7-fe22-40fa-a506-c5d70bad2677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:51.442566 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c6f7-fe22-40fa-a506-c5d70bad2677 None admin] REQUEST_METHOD: `PUT` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:51.442855 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c6f7-fe22-40fa-a506-c5d70bad2677 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:51.443138 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c6f7-fe22-40fa-a506-c5d70bad2677 None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/8824338846a940f0865b616ec42cce61/roles/b9c2d4c037024096bd7e864cea8b55e8` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:51.458819 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c6f7-fe22-40fa-a506-c5d70bad2677 None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=8824338846a940f0865b616ec42cce61, role_id=b9c2d4c037024096bd7e864cea8b55e8)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:51.460414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c6f7-fe22-40fa-a506-c5d70bad2677 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:51.579321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 615/1232] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:54:51 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/8824338846a940f0865b616ec42cce61/roles/b9c2d4c037024096bd7e864cea8b55e8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:54:53.709688 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90110d01-b4ae-4112-b930-56c0ea77e003 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:53.710759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90110d01-b4ae-4112-b930-56c0ea77e003 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:53.710759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90110d01-b4ae-4112-b930-56c0ea77e003 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:53.712128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 618/1233] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:54:53.717438 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893120a2-1465-4ccd-bc68-bbe2e77a09d1 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:53.718155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893120a2-1465-4ccd-bc68-bbe2e77a09d1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:53.718155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893120a2-1465-4ccd-bc68-bbe2e77a09d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:53.735186 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-893120a2-1465-4ccd-bc68-bbe2e77a09d1 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:54:53.736404 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-893120a2-1465-4ccd-bc68-bbe2e77a09d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:54:53.804833 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 616/1234] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:54:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:54:53.828617 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90110d01-b4ae-4112-b930-56c0ea77e003 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:53.855317 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:53.931691 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90110d01-b4ae-4112-b930-56c0ea77e003 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:53.965504 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b75e0f-b567-411d-bd4e-05097965773d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:53.966371 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b75e0f-b567-411d-bd4e-05097965773d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:53.966683 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b75e0f-b567-411d-bd4e-05097965773d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:53.966961 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b75e0f-b567-411d-bd4e-05097965773d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:53.977168 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b75e0f-b567-411d-bd4e-05097965773d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-a8b75e0f-b567-411d-bd4e-05097965773d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:54:53.985023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:53.986393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:54:53.987726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 619/1235] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 14:54:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:53.993384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893120a2-1465-4ccd-bc68-bbe2e77a09d1 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:54.039598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:54.092278 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893120a2-1465-4ccd-bc68-bbe2e77a09d1 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:54.125377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9071f-b725-4898-9f15-66a33d677f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:54.126041 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9071f-b725-4898-9f15-66a33d677f80 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:54.126331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9071f-b725-4898-9f15-66a33d677f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:54.126615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9071f-b725-4898-9f15-66a33d677f80 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:54.127366 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9071f-b725-4898-9f15-66a33d677f80 None admin] RBAC: Adding query filter params (name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:54.127682 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9071f-b725-4898-9f15-66a33d677f80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:54.129088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9071f-b725-4898-9f15-66a33d677f80 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:54.138208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 617/1236] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 14:54:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:54.144344 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b75e0f-b567-411d-bd4e-05097965773d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:54.231833 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:54.248020 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b75e0f-b567-411d-bd4e-05097965773d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:54.281644 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a9cf2-881b-4a48-bdee-8aca42037437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:54.282439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a9cf2-881b-4a48-bdee-8aca42037437 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:54.282726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a9cf2-881b-4a48-bdee-8aca42037437 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:54.283004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a9cf2-881b-4a48-bdee-8aca42037437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:54.294731 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a9cf2-881b-4a48-bdee-8aca42037437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:54.296185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a9cf2-881b-4a48-bdee-8aca42037437 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-6c7a9cf2-881b-4a48-bdee-8aca42037437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:54:54.313223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:54.314447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:54.315580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:54:54.315580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:54:54.315580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:54.315580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:54:54.315580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 620/1237] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:54.321482 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9071f-b725-4898-9f15-66a33d677f80 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:54.421722 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9071f-b725-4898-9f15-66a33d677f80 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:54.431551 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:54.456442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d68ac0-0311-4dbb-b4be-69f4e1b4cfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:54.457131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d68ac0-0311-4dbb-b4be-69f4e1b4cfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:54.457418 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d68ac0-0311-4dbb-b4be-69f4e1b4cfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:54.457705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d68ac0-0311-4dbb-b4be-69f4e1b4cfa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:54.458428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d68ac0-0311-4dbb-b4be-69f4e1b4cfa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:54.458765 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d68ac0-0311-4dbb-b4be-69f4e1b4cfa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:54.460179 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d68ac0-0311-4dbb-b4be-69f4e1b4cfa5 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:54.471877 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 618/1238] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:54.478164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a9cf2-881b-4a48-bdee-8aca42037437 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:54.584762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a9cf2-881b-4a48-bdee-8aca42037437 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:54.619682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84c057-fdf7-48a4-b016-69e65f484e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:54.620500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84c057-fdf7-48a4-b016-69e65f484e0e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:54.620783 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84c057-fdf7-48a4-b016-69e65f484e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:54.621058 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84c057-fdf7-48a4-b016-69e65f484e0e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:54.623810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:54.628792 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84c057-fdf7-48a4-b016-69e65f484e0e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:54.630230 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84c057-fdf7-48a4-b016-69e65f484e0e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-3f84c057-fdf7-48a4-b016-69e65f484e0e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:54:54.644892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:54.646413 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:54:54.647607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 621/1239] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:54.652164 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d68ac0-0311-4dbb-b4be-69f4e1b4cfa5 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:54.753297 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d68ac0-0311-4dbb-b4be-69f4e1b4cfa5 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:54.790006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dc1ed-eeae-4f88-a47d-2c60a74cd055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:54.790698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dc1ed-eeae-4f88-a47d-2c60a74cd055 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:54.790965 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dc1ed-eeae-4f88-a47d-2c60a74cd055 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:54.791293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dc1ed-eeae-4f88-a47d-2c60a74cd055 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:54.792143 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dc1ed-eeae-4f88-a47d-2c60a74cd055 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:54.792488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dc1ed-eeae-4f88-a47d-2c60a74cd055 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:54.793935 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dc1ed-eeae-4f88-a47d-2c60a74cd055 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:54.812100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 619/1240] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 14:54:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:54.818006 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84c057-fdf7-48a4-b016-69e65f484e0e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:54.919412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84c057-fdf7-48a4-b016-69e65f484e0e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:54.954223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73710c0-66c3-4bb3-9540-2b7a3f4708a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:54.954876 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73710c0-66c3-4bb3-9540-2b7a3f4708a2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:54.955166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73710c0-66c3-4bb3-9540-2b7a3f4708a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:54.955447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73710c0-66c3-4bb3-9540-2b7a3f4708a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:55.017603 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:55.025192 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73710c0-66c3-4bb3-9540-2b7a3f4708a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:55.026694 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73710c0-66c3-4bb3-9540-2b7a3f4708a2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:55.126051 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 622/1241] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:54:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:54:55.132091 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dc1ed-eeae-4f88-a47d-2c60a74cd055 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:55.160976 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:55.239204 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dc1ed-eeae-4f88-a47d-2c60a74cd055 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:55.275669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d6216-6682-4a26-b7a6-d1e5b797eb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:55.276418 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d6216-6682-4a26-b7a6-d1e5b797eb34 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:55.276710 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d6216-6682-4a26-b7a6-d1e5b797eb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:55.276996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d6216-6682-4a26-b7a6-d1e5b797eb34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:55.287188 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d6216-6682-4a26-b7a6-d1e5b797eb34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:55.289377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d6216-6682-4a26-b7a6-d1e5b797eb34 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-4d0d6216-6682-4a26-b7a6-d1e5b797eb34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:54:55.307089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:55.312448 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:55.313656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:54:55.313656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:54:55.313656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:55.313656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:55.313656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 620/1242] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:55.314181 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73710c0-66c3-4bb3-9540-2b7a3f4708a2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:55.414481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:55.420893 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73710c0-66c3-4bb3-9540-2b7a3f4708a2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:55.462503 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4651-36b5-4212-afa1-222f0c01478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:55.463183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4651-36b5-4212-afa1-222f0c01478e None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:55.463471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4651-36b5-4212-afa1-222f0c01478e None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:55.463780 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4651-36b5-4212-afa1-222f0c01478e None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:55.464547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e4651-36b5-4212-afa1-222f0c01478e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:55.464860 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e4651-36b5-4212-afa1-222f0c01478e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:55.466287 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e4651-36b5-4212-afa1-222f0c01478e None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:55.478530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 623/1243] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:55.484131 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d6216-6682-4a26-b7a6-d1e5b797eb34 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:55.568564 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:55.592291 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d6216-6682-4a26-b7a6-d1e5b797eb34 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:55.629303 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04f83f-684a-4485-b11f-27975aa8a881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:55.630149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04f83f-684a-4485-b11f-27975aa8a881 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:55.630439 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04f83f-684a-4485-b11f-27975aa8a881 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:55.630803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04f83f-684a-4485-b11f-27975aa8a881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:55.638664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04f83f-684a-4485-b11f-27975aa8a881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:55.640176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04f83f-684a-4485-b11f-27975aa8a881 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-df04f83f-684a-4485-b11f-27975aa8a881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:54:55.655561 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:55.660535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:55.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:54:55.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:54:55.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:54:55.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:54:55.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:54:55.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:54:55.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:54:55.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:55.661730 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 621/1244] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:55.662904 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4651-36b5-4212-afa1-222f0c01478e None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:55.767111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4651-36b5-4212-afa1-222f0c01478e None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:55.799850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9cf51-4485-415b-836f-11411cd9a704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:55.800490 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9cf51-4485-415b-836f-11411cd9a704 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:55.800772 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9cf51-4485-415b-836f-11411cd9a704 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:55.801047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9cf51-4485-415b-836f-11411cd9a704 None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:55.801821 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9cf51-4485-415b-836f-11411cd9a704 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:55.802128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9cf51-4485-415b-836f-11411cd9a704 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:55.803561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9cf51-4485-415b-836f-11411cd9a704 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:55.830938 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 624/1245] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:54:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:55.837215 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04f83f-684a-4485-b11f-27975aa8a881 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:55.942791 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04f83f-684a-4485-b11f-27975aa8a881 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:55.975840 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da7f26-f75d-47dc-b571-a029eef0a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:55.976525 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da7f26-f75d-47dc-b571-a029eef0a2ce None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:55.976852 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da7f26-f75d-47dc-b571-a029eef0a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:55.977095 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da7f26-f75d-47dc-b571-a029eef0a2ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:55.977923 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da7f26-f75d-47dc-b571-a029eef0a2ce None admin] RBAC: Adding query filter params (role.id=b9c2d4c037024096bd7e864cea8b55e8, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=8824338846a940f0865b616ec42cce61) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:55.978238 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da7f26-f75d-47dc-b571-a029eef0a2ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:55.979666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da7f26-f75d-47dc-b571-a029eef0a2ce None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:55.985430 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 622/1246] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:54:55 2022] GET /identity/v3/role_assignments?user.id=8824338846a940f0865b616ec42cce61&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=b9c2d4c037024096bd7e864cea8b55e8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:58.188911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef06be-0c43-4077-8adf-b4667f91db20 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:58.189482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef06be-0c43-4077-8adf-b4667f91db20 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:58.189482 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef06be-0c43-4077-8adf-b4667f91db20 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:58.190835 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 625/1247] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:54:58.197059 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea896772-5d96-45a9-a474-254edcf29878 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:58.197524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea896772-5d96-45a9-a474-254edcf29878 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:58.197640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea896772-5d96-45a9-a474-254edcf29878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:58.202678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:58.214592 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ea896772-5d96-45a9-a474-254edcf29878 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:54:58.215781 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ea896772-5d96-45a9-a474-254edcf29878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:54:58.283834 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 623/1248] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:54:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:54:58.308047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef06be-0c43-4077-8adf-b4667f91db20 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:58.327348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:58.411183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef06be-0c43-4077-8adf-b4667f91db20 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:58.446785 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324b1fb-61cf-4672-a7a3-7b0a706161a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:58.447504 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324b1fb-61cf-4672-a7a3-7b0a706161a5 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:58.447794 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324b1fb-61cf-4672-a7a3-7b0a706161a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:58.448072 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324b1fb-61cf-4672-a7a3-7b0a706161a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:58.458078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324b1fb-61cf-4672-a7a3-7b0a706161a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-d324b1fb-61cf-4672-a7a3-7b0a706161a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:54:58.466836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:58.472444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:54:58.473507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 626/1249] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:54:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:58.474426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea896772-5d96-45a9-a474-254edcf29878 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:58.576171 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea896772-5d96-45a9-a474-254edcf29878 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:58.607124 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:58.612057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6757c-ee6e-4692-bbb2-16d89fd64e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:58.612775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6757c-ee6e-4692-bbb2-16d89fd64e50 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:58.613058 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6757c-ee6e-4692-bbb2-16d89fd64e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:58.613331 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6757c-ee6e-4692-bbb2-16d89fd64e50 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:58.614078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6757c-ee6e-4692-bbb2-16d89fd64e50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:58.614628 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6757c-ee6e-4692-bbb2-16d89fd64e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:58.615744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6757c-ee6e-4692-bbb2-16d89fd64e50 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:58.624281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 624/1250] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:54:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:54:58.630152 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324b1fb-61cf-4672-a7a3-7b0a706161a5 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:58.708256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:58.732077 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324b1fb-61cf-4672-a7a3-7b0a706161a5 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:58.766270 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48513af-c09b-4337-b302-aac6cc4a3935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:58.767064 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48513af-c09b-4337-b302-aac6cc4a3935 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:58.767354 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48513af-c09b-4337-b302-aac6cc4a3935 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:58.767636 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48513af-c09b-4337-b302-aac6cc4a3935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:58.779779 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48513af-c09b-4337-b302-aac6cc4a3935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:58.781483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48513af-c09b-4337-b302-aac6cc4a3935 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-d48513af-c09b-4337-b302-aac6cc4a3935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:54:58.798865 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:58.799874 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:58.800941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:54:58.800941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:54:58.800941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:58.800941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:54:58.800941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 627/1251] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:58.806751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6757c-ee6e-4692-bbb2-16d89fd64e50 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:58.908158 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6757c-ee6e-4692-bbb2-16d89fd64e50 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:58.942015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95759bb3-259f-421d-871d-a633bedba35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:58.942669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95759bb3-259f-421d-871d-a633bedba35f None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:58.942947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95759bb3-259f-421d-871d-a633bedba35f None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:58.943221 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95759bb3-259f-421d-871d-a633bedba35f None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:58.943936 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95759bb3-259f-421d-871d-a633bedba35f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:58.944262 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95759bb3-259f-421d-871d-a633bedba35f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:58.946813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95759bb3-259f-421d-871d-a633bedba35f None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:58.958630 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 625/1252] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:58.964740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48513af-c09b-4337-b302-aac6cc4a3935 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:59.066464 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48513af-c09b-4337-b302-aac6cc4a3935 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:59.093926 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:59.102851 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dfb36-bd61-4ff9-8a3b-0177c0eb8c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:59.103721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dfb36-bd61-4ff9-8a3b-0177c0eb8c15 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:59.104005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dfb36-bd61-4ff9-8a3b-0177c0eb8c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:59.104310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dfb36-bd61-4ff9-8a3b-0177c0eb8c15 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:59.111761 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dfb36-bd61-4ff9-8a3b-0177c0eb8c15 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:59.113285 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dfb36-bd61-4ff9-8a3b-0177c0eb8c15 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-cc6dfb36-bd61-4ff9-8a3b-0177c0eb8c15 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:54:59.128576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:59.129740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:54:59.130836 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 628/1253] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:59.136336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95759bb3-259f-421d-871d-a633bedba35f None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:59.192177 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:59.237950 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95759bb3-259f-421d-871d-a633bedba35f None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:59.273675 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599cfe6-8964-444d-9e6b-f6c8419d6570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:59.274324 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599cfe6-8964-444d-9e6b-f6c8419d6570 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:59.274604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599cfe6-8964-444d-9e6b-f6c8419d6570 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:59.274881 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599cfe6-8964-444d-9e6b-f6c8419d6570 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:59.275716 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599cfe6-8964-444d-9e6b-f6c8419d6570 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:59.276024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599cfe6-8964-444d-9e6b-f6c8419d6570 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:59.277488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599cfe6-8964-444d-9e6b-f6c8419d6570 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:59.295445 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 626/1254] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 14:54:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:59.301163 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dfb36-bd61-4ff9-8a3b-0177c0eb8c15 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:59.401814 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dfb36-bd61-4ff9-8a3b-0177c0eb8c15 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:59.436831 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaf8a8-4faa-4a43-bcab-5a4c3cb40bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:59.437479 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaf8a8-4faa-4a43-bcab-5a4c3cb40bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:59.437774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaf8a8-4faa-4a43-bcab-5a4c3cb40bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:59.438055 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaf8a8-4faa-4a43-bcab-5a4c3cb40bd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:59.495758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:59.508174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaf8a8-4faa-4a43-bcab-5a4c3cb40bd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:59.509755 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaf8a8-4faa-4a43-bcab-5a4c3cb40bd4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:59.613349 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 629/1255] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:54:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:54:59.619784 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599cfe6-8964-444d-9e6b-f6c8419d6570 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:59.720368 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599cfe6-8964-444d-9e6b-f6c8419d6570 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:59.733330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:59.754106 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0119b-1cb9-496b-b8a7-9924f7059f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:59.754871 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0119b-1cb9-496b-b8a7-9924f7059f50 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:59.755157 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0119b-1cb9-496b-b8a7-9924f7059f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:59.755430 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0119b-1cb9-496b-b8a7-9924f7059f50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:59.766229 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0119b-1cb9-496b-b8a7-9924f7059f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:59.767667 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0119b-1cb9-496b-b8a7-9924f7059f50 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-a3d0119b-1cb9-496b-b8a7-9924f7059f50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:54:59.786006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:54:59.787067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:54:59.788118 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:54:59.788118 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:54:59.788118 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:54:59.788118 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:54:59.791403 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 627/1256] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:54:59.793891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaf8a8-4faa-4a43-bcab-5a4c3cb40bd4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:54:59.881111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:54:59.893208 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaf8a8-4faa-4a43-bcab-5a4c3cb40bd4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:54:59.928062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503ecab-d988-4281-8b3c-215328d10d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:54:59.928803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503ecab-d988-4281-8b3c-215328d10d54 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:54:59.929124 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503ecab-d988-4281-8b3c-215328d10d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:54:59.930092 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503ecab-d988-4281-8b3c-215328d10d54 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:54:59.931139 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503ecab-d988-4281-8b3c-215328d10d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:54:59.931507 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503ecab-d988-4281-8b3c-215328d10d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:54:59.933014 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503ecab-d988-4281-8b3c-215328d10d54 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:54:59.944193 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 630/1257] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:54:59.950102 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0119b-1cb9-496b-b8a7-9924f7059f50 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:00.051680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0119b-1cb9-496b-b8a7-9924f7059f50 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:00.086640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa7ae0-7fc4-4ea8-be3a-1a2e856154b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:00.087429 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa7ae0-7fc4-4ea8-be3a-1a2e856154b9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:00.087717 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa7ae0-7fc4-4ea8-be3a-1a2e856154b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:00.087995 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa7ae0-7fc4-4ea8-be3a-1a2e856154b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:00.094699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa7ae0-7fc4-4ea8-be3a-1a2e856154b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:00.096133 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa7ae0-7fc4-4ea8-be3a-1a2e856154b9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-dbaa7ae0-7fc4-4ea8-be3a-1a2e856154b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:55:00.110409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:00.111481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:00.112678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:55:00.112678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:55:00.112678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:55:00.112678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:55:00.112678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:55:00.112678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:55:00.112678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:55:00.112678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:00.112678 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 628/1258] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:54:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:00.118514 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503ecab-d988-4281-8b3c-215328d10d54 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:00.218074 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503ecab-d988-4281-8b3c-215328d10d54 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:00.252053 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a425ff4-f1e0-45b1-a41e-12ed3e626d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:00.252742 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a425ff4-f1e0-45b1-a41e-12ed3e626d3c None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:00.253030 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a425ff4-f1e0-45b1-a41e-12ed3e626d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:00.253309 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a425ff4-f1e0-45b1-a41e-12ed3e626d3c None admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:00.254102 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a425ff4-f1e0-45b1-a41e-12ed3e626d3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:00.254415 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a425ff4-f1e0-45b1-a41e-12ed3e626d3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:00.255908 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a425ff4-f1e0-45b1-a41e-12ed3e626d3c None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:00.268673 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 631/1259] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:55:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:00.275502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa7ae0-7fc4-4ea8-be3a-1a2e856154b9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:00.303119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:00.376896 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa7ae0-7fc4-4ea8-be3a-1a2e856154b9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:00.411178 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9382d54-5bb7-4447-8dbf-31767e0441d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:00.411820 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9382d54-5bb7-4447-8dbf-31767e0441d7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:00.412110 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9382d54-5bb7-4447-8dbf-31767e0441d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:00.412424 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9382d54-5bb7-4447-8dbf-31767e0441d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:00.413258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9382d54-5bb7-4447-8dbf-31767e0441d7 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=8824338846a940f0865b616ec42cce61) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:00.413576 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9382d54-5bb7-4447-8dbf-31767e0441d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:00.415775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9382d54-5bb7-4447-8dbf-31767e0441d7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:00.421332 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 629/1260] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:55:00 2022] GET /identity/v3/role_assignments?user.id=8824338846a940f0865b616ec42cce61&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:02.583295 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c627c-6e6d-4a98-9e76-b544b7796e80 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:02.583854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c627c-6e6d-4a98-9e76-b544b7796e80 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:02.583854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c627c-6e6d-4a98-9e76-b544b7796e80 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:02.585258 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 632/1261] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:02.590818 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19038701-6c31-4a78-bcfe-557bc4ff302a None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:02.591239 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19038701-6c31-4a78-bcfe-557bc4ff302a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:02.591380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19038701-6c31-4a78-bcfe-557bc4ff302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:02.608009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-19038701-6c31-4a78-bcfe-557bc4ff302a None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:02.609233 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-19038701-6c31-4a78-bcfe-557bc4ff302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:02.675313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 630/1262] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:55:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:55:02.699162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c627c-6e6d-4a98-9e76-b544b7796e80 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:02.709958 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:02.799509 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c627c-6e6d-4a98-9e76-b544b7796e80 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:02.835753 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ac66-1b98-440a-809a-f2301e323d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:02.836641 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ac66-1b98-440a-809a-f2301e323d97 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:02.836933 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ac66-1b98-440a-809a-f2301e323d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:02.837240 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ac66-1b98-440a-809a-f2301e323d97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:02.847492 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928ac66-1b98-440a-809a-f2301e323d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1928ac66-1b98-440a-809a-f2301e323d97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:55:02.855495 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:02.862992 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:55:02.864056 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 633/1263] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:55:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:02.865103 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19038701-6c31-4a78-bcfe-557bc4ff302a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:02.881918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:02.977167 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19038701-6c31-4a78-bcfe-557bc4ff302a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:03.013375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76545d-d2c6-4dae-bb65-f2a5fa980585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:03.014039 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76545d-d2c6-4dae-bb65-f2a5fa980585 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:03.014322 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76545d-d2c6-4dae-bb65-f2a5fa980585 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:03.014599 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76545d-d2c6-4dae-bb65-f2a5fa980585 None admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:03.015343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76545d-d2c6-4dae-bb65-f2a5fa980585 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:03.015652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76545d-d2c6-4dae-bb65-f2a5fa980585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:03.017116 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76545d-d2c6-4dae-bb65-f2a5fa980585 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:03.025700 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 631/1264] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:55:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:03.031524 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ac66-1b98-440a-809a-f2301e323d97 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:03.100996 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:03.135461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ac66-1b98-440a-809a-f2301e323d97 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:03.167445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa2d31-851a-42d0-a06d-2c1e5b83b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:03.168256 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2d31-851a-42d0-a06d-2c1e5b83b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:03.168542 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2d31-851a-42d0-a06d-2c1e5b83b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:03.168820 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2d31-851a-42d0-a06d-2c1e5b83b7b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:03.180454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa2d31-851a-42d0-a06d-2c1e5b83b7b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:03.181884 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa2d31-851a-42d0-a06d-2c1e5b83b7b8 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-fcaa2d31-851a-42d0-a06d-2c1e5b83b7b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:55:03.199125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:03.200253 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:03.201575 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:55:03.201575 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:55:03.201575 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:55:03.201575 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:55:03.201575 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 634/1265] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:03.206248 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76545d-d2c6-4dae-bb65-f2a5fa980585 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:03.291086 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:03.309752 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76545d-d2c6-4dae-bb65-f2a5fa980585 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:03.344240 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0641168-dd55-4226-9bd0-0896b4f968a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:03.345008 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641168-dd55-4226-9bd0-0896b4f968a9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:03.345293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641168-dd55-4226-9bd0-0896b4f968a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:03.345571 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641168-dd55-4226-9bd0-0896b4f968a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:03.346290 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0641168-dd55-4226-9bd0-0896b4f968a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:03.346619 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0641168-dd55-4226-9bd0-0896b4f968a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:03.347974 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0641168-dd55-4226-9bd0-0896b4f968a9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:03.369357 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 632/1266] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:03.369608 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa2d31-851a-42d0-a06d-2c1e5b83b7b8 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:03.467401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa2d31-851a-42d0-a06d-2c1e5b83b7b8 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:03.487035 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:03.501282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2185fd1-2221-4e6f-b38e-24f5cbe50655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:03.502079 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2185fd1-2221-4e6f-b38e-24f5cbe50655 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:03.502366 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2185fd1-2221-4e6f-b38e-24f5cbe50655 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:03.502648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2185fd1-2221-4e6f-b38e-24f5cbe50655 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:03.510659 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2185fd1-2221-4e6f-b38e-24f5cbe50655 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:03.512134 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2185fd1-2221-4e6f-b38e-24f5cbe50655 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-f2185fd1-2221-4e6f-b38e-24f5cbe50655 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:55:03.526943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:03.528150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:55:03.529397 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 635/1267] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:55:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:03.534111 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0641168-dd55-4226-9bd0-0896b4f968a9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:03.634619 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0641168-dd55-4226-9bd0-0896b4f968a9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:03.669460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ceb84f-35f2-457c-9c68-77aa1d3b51c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:03.670142 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ceb84f-35f2-457c-9c68-77aa1d3b51c7 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:03.670431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ceb84f-35f2-457c-9c68-77aa1d3b51c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:03.670713 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ceb84f-35f2-457c-9c68-77aa1d3b51c7 None admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:03.671607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ceb84f-35f2-457c-9c68-77aa1d3b51c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:03.671915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ceb84f-35f2-457c-9c68-77aa1d3b51c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:03.673529 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ceb84f-35f2-457c-9c68-77aa1d3b51c7 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:03.683380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:03.691629 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 633/1268] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 14:55:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:55:03.697789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2185fd1-2221-4e6f-b38e-24f5cbe50655 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:03.798290 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2185fd1-2221-4e6f-b38e-24f5cbe50655 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:03.830810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254feaa-5426-4535-b5d8-92f926cfd070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:03.831598 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254feaa-5426-4535-b5d8-92f926cfd070 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:03.831887 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254feaa-5426-4535-b5d8-92f926cfd070 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:03.832167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254feaa-5426-4535-b5d8-92f926cfd070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:03.843080 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254feaa-5426-4535-b5d8-92f926cfd070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:03.844660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254feaa-5426-4535-b5d8-92f926cfd070 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-1254feaa-5426-4535-b5d8-92f926cfd070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:55:03.862133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:03.863259 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:03.864183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:55:03.864183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:55:03.864183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:55:03.864183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:55:03.864183 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 636/1269] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:03.869313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ceb84f-35f2-457c-9c68-77aa1d3b51c7 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:03.969385 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ceb84f-35f2-457c-9c68-77aa1d3b51c7 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:04.002978 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ed8be-7da6-43a1-95a2-66a3d9e7827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:04.003640 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ed8be-7da6-43a1-95a2-66a3d9e7827e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:04.003922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ed8be-7da6-43a1-95a2-66a3d9e7827e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:04.004203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ed8be-7da6-43a1-95a2-66a3d9e7827e None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:04.004971 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ed8be-7da6-43a1-95a2-66a3d9e7827e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:04.005262 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ed8be-7da6-43a1-95a2-66a3d9e7827e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:04.006627 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ed8be-7da6-43a1-95a2-66a3d9e7827e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:04.018469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 634/1270] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:55:04.025312 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254feaa-5426-4535-b5d8-92f926cfd070 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:04.057121 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:04.124041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254feaa-5426-4535-b5d8-92f926cfd070 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:04.161387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5d956-5a70-4c69-bd35-9a8422e6842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:04.162178 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5d956-5a70-4c69-bd35-9a8422e6842d None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:04.162465 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5d956-5a70-4c69-bd35-9a8422e6842d None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:04.162744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5d956-5a70-4c69-bd35-9a8422e6842d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:04.170081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5d956-5a70-4c69-bd35-9a8422e6842d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:04.171550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5d956-5a70-4c69-bd35-9a8422e6842d None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-86c5d956-5a70-4c69-bd35-9a8422e6842d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:55:04.187707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:04.189005 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:04.190243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:55:04.190243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:55:04.190243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:55:04.190243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:55:04.190243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:55:04.190243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:55:04.190243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:55:04.190243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:55:04.190243 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 637/1271] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:55:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:04.195308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ed8be-7da6-43a1-95a2-66a3d9e7827e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:04.265368 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:04.313036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ed8be-7da6-43a1-95a2-66a3d9e7827e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:04.351057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e0693-62ee-409f-b0db-ffc67a86e7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:04.352006 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e0693-62ee-409f-b0db-ffc67a86e7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:04.352596 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e0693-62ee-409f-b0db-ffc67a86e7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:04.352967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e0693-62ee-409f-b0db-ffc67a86e7f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:04.353631 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e0693-62ee-409f-b0db-ffc67a86e7f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:04.353887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e0693-62ee-409f-b0db-ffc67a86e7f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:04.355340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e0693-62ee-409f-b0db-ffc67a86e7f1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:04.370207 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 635/1272] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:04.377500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5d956-5a70-4c69-bd35-9a8422e6842d None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:04.475775 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:04.486359 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5d956-5a70-4c69-bd35-9a8422e6842d None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:04.530744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558efc6-25f8-46a8-a886-4d11322c727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:04.530744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558efc6-25f8-46a8-a886-4d11322c727b None admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:04.530744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558efc6-25f8-46a8-a886-4d11322c727b None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:04.530744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558efc6-25f8-46a8-a886-4d11322c727b None admin] PATH_INFO: `/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/8824338846a940f0865b616ec42cce61/roles/dbdf8e124c2f476abf41ccc73efd25cc` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:04.531307 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558efc6-25f8-46a8-a886-4d11322c727b None admin] RBAC: Authorizing `identity:create_grant(project_id=7571fbc040844bc29d293fde55eb53ad, user_id=8824338846a940f0865b616ec42cce61, role_id=dbdf8e124c2f476abf41ccc73efd25cc)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:04.531307 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558efc6-25f8-46a8-a886-4d11322c727b None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:04.657739 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 638/1273] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 14:55:04 2022] PUT /identity/v3/projects/7571fbc040844bc29d293fde55eb53ad/users/8824338846a940f0865b616ec42cce61/roles/dbdf8e124c2f476abf41ccc73efd25cc => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:55:06.790089 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992f005-9e43-4d44-adf7-740a819faa9e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:06.791466 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992f005-9e43-4d44-adf7-740a819faa9e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:06.791466 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992f005-9e43-4d44-adf7-740a819faa9e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:06.792121 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 636/1274] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:06.797574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf20eae-d8f7-48d8-8179-a10ca4507cbb None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:06.798002 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf20eae-d8f7-48d8-8179-a10ca4507cbb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:06.798172 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf20eae-d8f7-48d8-8179-a10ca4507cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:06.815517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6cf20eae-d8f7-48d8-8179-a10ca4507cbb None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:06.816762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6cf20eae-d8f7-48d8-8179-a10ca4507cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:06.891921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 639/1275] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:55:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:55:06.916052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992f005-9e43-4d44-adf7-740a819faa9e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:07.028371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992f005-9e43-4d44-adf7-740a819faa9e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:07.047887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:07.065024 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c25f2-6611-49ef-8c64-fd302dd16a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:07.065813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c25f2-6611-49ef-8c64-fd302dd16a91 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:07.066093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c25f2-6611-49ef-8c64-fd302dd16a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:07.066465 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c25f2-6611-49ef-8c64-fd302dd16a91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:07.077077 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c25f2-6611-49ef-8c64-fd302dd16a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-ba9c25f2-6611-49ef-8c64-fd302dd16a91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:55:07.084876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:07.086104 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:07.087100 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 637/1276] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 14:55:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:07.092419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf20eae-d8f7-48d8-8179-a10ca4507cbb None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:07.177612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:07.196872 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf20eae-d8f7-48d8-8179-a10ca4507cbb None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:07.230959 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883ea6d-4643-4f09-b625-e18a67d9947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:07.231830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883ea6d-4643-4f09-b625-e18a67d9947a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:07.232114 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883ea6d-4643-4f09-b625-e18a67d9947a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:07.232422 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883ea6d-4643-4f09-b625-e18a67d9947a None admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:07.233174 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883ea6d-4643-4f09-b625-e18a67d9947a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:07.233486 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883ea6d-4643-4f09-b625-e18a67d9947a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:07.234868 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883ea6d-4643-4f09-b625-e18a67d9947a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:07.244096 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 640/1277] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 14:55:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:07.250078 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c25f2-6611-49ef-8c64-fd302dd16a91 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:07.355455 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c25f2-6611-49ef-8c64-fd302dd16a91 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:07.390446 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d87bb9-539b-49bf-9974-031ebab1c5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:07.391230 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d87bb9-539b-49bf-9974-031ebab1c5ef None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:07.391512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d87bb9-539b-49bf-9974-031ebab1c5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:07.391814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d87bb9-539b-49bf-9974-031ebab1c5ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:07.403542 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d87bb9-539b-49bf-9974-031ebab1c5ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:07.405015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d87bb9-539b-49bf-9974-031ebab1c5ef None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-46d87bb9-539b-49bf-9974-031ebab1c5ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:55:07.423375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:07.432052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:07.433342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:55:07.433342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:55:07.433342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:55:07.433342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:07.433342 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 638/1278] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:07.433707 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883ea6d-4643-4f09-b625-e18a67d9947a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:07.536160 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883ea6d-4643-4f09-b625-e18a67d9947a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:07.570441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c546305-e4a8-4195-9eb8-cd2ef9345aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:07.571111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c546305-e4a8-4195-9eb8-cd2ef9345aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:07.571407 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c546305-e4a8-4195-9eb8-cd2ef9345aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:07.571686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c546305-e4a8-4195-9eb8-cd2ef9345aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:07.572446 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c546305-e4a8-4195-9eb8-cd2ef9345aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:07.572755 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c546305-e4a8-4195-9eb8-cd2ef9345aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:07.574144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c546305-e4a8-4195-9eb8-cd2ef9345aa4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:07.586967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 641/1279] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:55:07.592944 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d87bb9-539b-49bf-9974-031ebab1c5ef None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:07.625948 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:07.698575 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d87bb9-539b-49bf-9974-031ebab1c5ef None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:07.734515 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2e99f-c5e5-4a71-80e9-62b8f6823cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:07.735353 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2e99f-c5e5-4a71-80e9-62b8f6823cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:07.735856 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2e99f-c5e5-4a71-80e9-62b8f6823cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:07.736348 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2e99f-c5e5-4a71-80e9-62b8f6823cc9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:07.744744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2e99f-c5e5-4a71-80e9-62b8f6823cc9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:07.746191 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2e99f-c5e5-4a71-80e9-62b8f6823cc9 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-72c2e99f-c5e5-4a71-80e9-62b8f6823cc9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:55:07.761521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:07.762702 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:07.764173 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 639/1280] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:55:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:07.768795 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c546305-e4a8-4195-9eb8-cd2ef9345aa4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:07.772700 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:07.870242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c546305-e4a8-4195-9eb8-cd2ef9345aa4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:07.905514 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de54bd-5dba-46ac-a597-03904d38c0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:07.906172 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de54bd-5dba-46ac-a597-03904d38c0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:07.906462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de54bd-5dba-46ac-a597-03904d38c0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:07.906746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de54bd-5dba-46ac-a597-03904d38c0b4 None admin] PATH_INFO: `/v3/users` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:07.907583 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de54bd-5dba-46ac-a597-03904d38c0b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:07.907899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de54bd-5dba-46ac-a597-03904d38c0b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:07.909409 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de54bd-5dba-46ac-a597-03904d38c0b4 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:07.926983 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 642/1281] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 14:55:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:07.933339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2e99f-c5e5-4a71-80e9-62b8f6823cc9 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:08.027699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:08.041958 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2e99f-c5e5-4a71-80e9-62b8f6823cc9 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:08.076882 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281a33d-2a13-4cda-ae1b-cc88b8f3f75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:08.077537 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281a33d-2a13-4cda-ae1b-cc88b8f3f75c None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:08.077827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281a33d-2a13-4cda-ae1b-cc88b8f3f75c None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:08.078114 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281a33d-2a13-4cda-ae1b-cc88b8f3f75c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:08.147901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281a33d-2a13-4cda-ae1b-cc88b8f3f75c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:08.149386 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281a33d-2a13-4cda-ae1b-cc88b8f3f75c None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:08.244921 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:08.255351 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 640/1282] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 14:55:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:55:08.262111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de54bd-5dba-46ac-a597-03904d38c0b4 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:08.332657 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:08.372206 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de54bd-5dba-46ac-a597-03904d38c0b4 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:08.407412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccf985-ea28-47db-b37e-3b336bb2dd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:08.408194 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccf985-ea28-47db-b37e-3b336bb2dd1a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:08.408502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccf985-ea28-47db-b37e-3b336bb2dd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:08.408796 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccf985-ea28-47db-b37e-3b336bb2dd1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:08.419062 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ccf985-ea28-47db-b37e-3b336bb2dd1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:08.420610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ccf985-ea28-47db-b37e-3b336bb2dd1a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-d7ccf985-ea28-47db-b37e-3b336bb2dd1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:55:08.439136 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:08.440282 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:08.441454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:55:08.441454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:55:08.441454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:55:08.441454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:55:08.441454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 643/1283] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 14:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:08.447185 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281a33d-2a13-4cda-ae1b-cc88b8f3f75c None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:08.555071 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281a33d-2a13-4cda-ae1b-cc88b8f3f75c None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:08.594093 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114497ff-b292-43a1-9c14-c6606d84de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:08.594756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114497ff-b292-43a1-9c14-c6606d84de5e None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:08.595045 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114497ff-b292-43a1-9c14-c6606d84de5e None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:08.595320 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114497ff-b292-43a1-9c14-c6606d84de5e None admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:08.596038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114497ff-b292-43a1-9c14-c6606d84de5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:08.596378 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114497ff-b292-43a1-9c14-c6606d84de5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:08.597761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114497ff-b292-43a1-9c14-c6606d84de5e None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:08.610274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 641/1284] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 14:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:55:08.616111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccf985-ea28-47db-b37e-3b336bb2dd1a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:08.718303 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccf985-ea28-47db-b37e-3b336bb2dd1a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:08.733194 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:08.753481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4437-a9df-4470-a930-4ef7a38043b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:08.754272 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4437-a9df-4470-a930-4ef7a38043b2 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:08.754553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4437-a9df-4470-a930-4ef7a38043b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:08.754826 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4437-a9df-4470-a930-4ef7a38043b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:08.762592 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e4437-a9df-4470-a930-4ef7a38043b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:08.764063 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e4437-a9df-4470-a930-4ef7a38043b2 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application [None req-706e4437-a9df-4470-a930-4ef7a38043b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:55:08.778553 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:08.779639 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:08.780840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:55:08.780840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:55:08.780840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:55:08.780840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:55:08.780840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:55:08.780840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:55:08.780840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:55:08.780840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: ERROR keystone.server.flask.application Jun 16 14:55:08.780840 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 644/1285] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 14:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:08.785688 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114497ff-b292-43a1-9c14-c6606d84de5e None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:08.839305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:08.889711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114497ff-b292-43a1-9c14-c6606d84de5e None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:08.924181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fbacc-6242-467e-a623-23e55d2f05d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:08.924870 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fbacc-6242-467e-a623-23e55d2f05d8 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:08.925161 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fbacc-6242-467e-a623-23e55d2f05d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:08.925444 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fbacc-6242-467e-a623-23e55d2f05d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:08.926293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fbacc-6242-467e-a623-23e55d2f05d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:08.926604 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fbacc-6242-467e-a623-23e55d2f05d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:08.928029 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fbacc-6242-467e-a623-23e55d2f05d8 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:08.939997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 642/1286] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 14:55:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:08.947167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4437-a9df-4470-a930-4ef7a38043b2 None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:09.053041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4437-a9df-4470-a930-4ef7a38043b2 None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:09.088115 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7042e0a-7c70-41e2-b76b-7a74541b5780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:09.088806 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7042e0a-7c70-41e2-b76b-7a74541b5780 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:09.089099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7042e0a-7c70-41e2-b76b-7a74541b5780 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:09.089381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7042e0a-7c70-41e2-b76b-7a74541b5780 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:09.090284 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7042e0a-7c70-41e2-b76b-7a74541b5780 None admin] RBAC: Adding query filter params (role.id=dbdf8e124c2f476abf41ccc73efd25cc, scope.project.id=7571fbc040844bc29d293fde55eb53ad, user.id=8824338846a940f0865b616ec42cce61) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:09.090609 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7042e0a-7c70-41e2-b76b-7a74541b5780 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:09.092127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7042e0a-7c70-41e2-b76b-7a74541b5780 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:09.098348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 645/1287] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 14:55:08 2022] GET /identity/v3/role_assignments?user.id=8824338846a940f0865b616ec42cce61&scope.project.id=7571fbc040844bc29d293fde55eb53ad&role.id=dbdf8e124c2f476abf41ccc73efd25cc => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:11.317711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9fc0-2751-4e77-82e0-f5b5abdcd4f0 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:11.318279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9fc0-2751-4e77-82e0-f5b5abdcd4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:11.318279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9fc0-2751-4e77-82e0-f5b5abdcd4f0 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:11.319694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 643/1288] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:11.324999 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518514af-1f44-4066-86db-e40bf4529d3d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:11.325470 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518514af-1f44-4066-86db-e40bf4529d3d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:11.325610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518514af-1f44-4066-86db-e40bf4529d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:11.343920 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-518514af-1f44-4066-86db-e40bf4529d3d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:11.345169 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-518514af-1f44-4066-86db-e40bf4529d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:11.377371 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:11.416028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 646/1289] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:55:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:55:11.439156 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9fc0-2751-4e77-82e0-f5b5abdcd4f0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:11.550375 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9fc0-2751-4e77-82e0-f5b5abdcd4f0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:11.561536 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:11.585277 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9c53d-7aa9-4be3-9e4b-51abcce73ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:11.586647 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9c53d-7aa9-4be3-9e4b-51abcce73ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:11.586931 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9c53d-7aa9-4be3-9e4b-51abcce73ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:11.587206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9c53d-7aa9-4be3-9e4b-51abcce73ae2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:11.587897 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9c53d-7aa9-4be3-9e4b-51abcce73ae2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:11.589397 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9c53d-7aa9-4be3-9e4b-51abcce73ae2 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-32c9c53d-7aa9-4be3-9e4b-51abcce73ae2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:11.601168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:11.602835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:11.604096 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 644/1290] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:55:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:11.608403 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518514af-1f44-4066-86db-e40bf4529d3d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:11.709746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518514af-1f44-4066-86db-e40bf4529d3d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:11.744476 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf9b19-8514-40b1-bc11-70e3966e3bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:11.745143 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf9b19-8514-40b1-bc11-70e3966e3bde None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:11.745431 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf9b19-8514-40b1-bc11-70e3966e3bde None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:11.745712 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf9b19-8514-40b1-bc11-70e3966e3bde None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:11.746499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf9b19-8514-40b1-bc11-70e3966e3bde None admin] RBAC: Adding query filter params (name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:11.746827 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf9b19-8514-40b1-bc11-70e3966e3bde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:11.748247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf9b19-8514-40b1-bc11-70e3966e3bde None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:11.753505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 647/1291] 10.222.0.88 () {56 vars in 1116 bytes} [Thu Jun 16 14:55:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:11.760411 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9c53d-7aa9-4be3-9e4b-51abcce73ae2 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:11.861857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9c53d-7aa9-4be3-9e4b-51abcce73ae2 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:11.895843 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0c62f-e769-4e00-9a9b-0d56f3f5f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:11.896594 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0c62f-e769-4e00-9a9b-0d56f3f5f008 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:11.896885 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0c62f-e769-4e00-9a9b-0d56f3f5f008 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:11.897195 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0c62f-e769-4e00-9a9b-0d56f3f5f008 None admin] PATH_INFO: `/v3/services` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:11.897922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0c62f-e769-4e00-9a9b-0d56f3f5f008 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:11.898235 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0c62f-e769-4e00-9a9b-0d56f3f5f008 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:11.899705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0c62f-e769-4e00-9a9b-0d56f3f5f008 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:11.904240 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 645/1292] 10.222.0.88 () {56 vars in 1116 bytes} [Thu Jun 16 14:55:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:14.091825 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230fa5e-6691-421d-8b48-bead5e9aecf8 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:14.092426 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230fa5e-6691-421d-8b48-bead5e9aecf8 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:14.092426 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230fa5e-6691-421d-8b48-bead5e9aecf8 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:14.093839 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 648/1293] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:55:14.099363 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d59389-3207-4b36-93af-ff269ba15245 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:14.099801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d59389-3207-4b36-93af-ff269ba15245 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:14.099913 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d59389-3207-4b36-93af-ff269ba15245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:14.117947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a5d59389-3207-4b36-93af-ff269ba15245 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:14.119219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a5d59389-3207-4b36-93af-ff269ba15245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:14.151555 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:14.190620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 646/1294] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:55:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:55:14.215449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230fa5e-6691-421d-8b48-bead5e9aecf8 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:14.247730 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:14.320554 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230fa5e-6691-421d-8b48-bead5e9aecf8 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:14.355839 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7f90f-7943-451d-ac0f-a1456f2970f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:14.356478 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7f90f-7943-451d-ac0f-a1456f2970f1 None admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:14.356767 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7f90f-7943-451d-ac0f-a1456f2970f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:14.357046 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7f90f-7943-451d-ac0f-a1456f2970f1 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:14.357781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7f90f-7943-451d-ac0f-a1456f2970f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:14.359190 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7f90f-7943-451d-ac0f-a1456f2970f1 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:14.444203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 649/1295] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 14:55:14 2022] POST /identity/v3/services => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:55:16.697415 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4998b2-3ad4-4632-8ca7-1321f9b4e82e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:16.698239 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4998b2-3ad4-4632-8ca7-1321f9b4e82e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:16.698239 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4998b2-3ad4-4632-8ca7-1321f9b4e82e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:16.699504 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 647/1296] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:16.707384 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e8e9f-e25a-48c0-a051-9961f6dfb649 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:16.707937 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e8e9f-e25a-48c0-a051-9961f6dfb649 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:16.707937 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e8e9f-e25a-48c0-a051-9961f6dfb649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:16.728334 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d20e8e9f-e25a-48c0-a051-9961f6dfb649 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:16.733176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d20e8e9f-e25a-48c0-a051-9961f6dfb649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:16.808463 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 650/1297] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:55:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:55:16.831210 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4998b2-3ad4-4632-8ca7-1321f9b4e82e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:16.942144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4998b2-3ad4-4632-8ca7-1321f9b4e82e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:16.976990 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71492fc-022f-46ca-a18b-2cb418809b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:16.977788 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71492fc-022f-46ca-a18b-2cb418809b47 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:16.978074 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71492fc-022f-46ca-a18b-2cb418809b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:16.978358 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71492fc-022f-46ca-a18b-2cb418809b47 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:16.979009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71492fc-022f-46ca-a18b-2cb418809b47 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:16.980409 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71492fc-022f-46ca-a18b-2cb418809b47 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-d71492fc-022f-46ca-a18b-2cb418809b47 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:16.992748 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:16.994060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:16.994998 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 648/1298] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:55:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:16.999984 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e8e9f-e25a-48c0-a051-9961f6dfb649 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:17.019100 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:17.105246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e8e9f-e25a-48c0-a051-9961f6dfb649 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:17.138942 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eee19a-c8b3-4903-840d-73369d6e266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:17.139605 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eee19a-c8b3-4903-840d-73369d6e266a None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:17.139892 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eee19a-c8b3-4903-840d-73369d6e266a None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:17.140171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eee19a-c8b3-4903-840d-73369d6e266a None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:17.140945 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eee19a-c8b3-4903-840d-73369d6e266a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:17.141251 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eee19a-c8b3-4903-840d-73369d6e266a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:17.142646 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eee19a-c8b3-4903-840d-73369d6e266a None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:17.149083 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 651/1299] 10.222.0.88 () {56 vars in 1116 bytes} [Thu Jun 16 14:55:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:17.155163 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71492fc-022f-46ca-a18b-2cb418809b47 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:17.188834 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:17.260680 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71492fc-022f-46ca-a18b-2cb418809b47 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:17.294187 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6f9a8-f230-4c3a-b114-76833fbd5ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:17.294839 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6f9a8-f230-4c3a-b114-76833fbd5ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:17.295127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6f9a8-f230-4c3a-b114-76833fbd5ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:17.295410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6f9a8-f230-4c3a-b114-76833fbd5ad1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:17.296198 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6f9a8-f230-4c3a-b114-76833fbd5ad1 None admin] RBAC: Adding query filter params (interface=public, service_id=c08a9f7e0d41406d848dc62dbf674a79, region_id=RegionOne) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:17.296549 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6f9a8-f230-4c3a-b114-76833fbd5ad1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:17.298057 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6f9a8-f230-4c3a-b114-76833fbd5ad1 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:17.303460 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 649/1300] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 14:55:17 2022] GET /identity/v3/endpoints?service_id=c08a9f7e0d41406d848dc62dbf674a79&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:17.310685 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eee19a-c8b3-4903-840d-73369d6e266a None admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:17.412188 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eee19a-c8b3-4903-840d-73369d6e266a None admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:17.447473 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766acf5-c8c6-4536-ae2d-3a10ed31dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:17.448151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766acf5-c8c6-4536-ae2d-3a10ed31dd70 None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:17.448471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766acf5-c8c6-4536-ae2d-3a10ed31dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:17.448758 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766acf5-c8c6-4536-ae2d-3a10ed31dd70 None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:17.449460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766acf5-c8c6-4536-ae2d-3a10ed31dd70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:17.450880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766acf5-c8c6-4536-ae2d-3a10ed31dd70 None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:17.458296 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 652/1301] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 14:55:17 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:55:17.581950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:19.668382 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c054435-df30-42f2-aa8e-76c05bcd2f19 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:19.669008 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c054435-df30-42f2-aa8e-76c05bcd2f19 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:19.669208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c054435-df30-42f2-aa8e-76c05bcd2f19 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:19.671244 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 650/1302] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 14:55:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:19.676822 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f62a6-c123-468f-8460-41818f9d6fad None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:19.677325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f62a6-c123-468f-8460-41818f9d6fad None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:19.677416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f62a6-c123-468f-8460-41818f9d6fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:19.695426 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-746f62a6-c123-468f-8460-41818f9d6fad None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:19.696803 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-746f62a6-c123-468f-8460-41818f9d6fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:19.770133 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 653/1303] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 14:55:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:55:19.793416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c054435-df30-42f2-aa8e-76c05bcd2f19 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:19.900142 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:19.907354 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c054435-df30-42f2-aa8e-76c05bcd2f19 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:19.947831 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afe69b-0295-4a92-bcf4-de547310d536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:19.948668 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afe69b-0295-4a92-bcf4-de547310d536 None admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:19.948960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afe69b-0295-4a92-bcf4-de547310d536 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:19.949241 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afe69b-0295-4a92-bcf4-de547310d536 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:19.949912 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afe69b-0295-4a92-bcf4-de547310d536 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:19.951293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afe69b-0295-4a92-bcf4-de547310d536 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application [None req-b2afe69b-0295-4a92-bcf4-de547310d536 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:55:19.963521 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:55:19.964987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application with Lock( Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: ERROR keystone.server.flask.application Jun 16 14:55:19.966751 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 651/1304] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 14:55:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:55:19.972789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f62a6-c123-468f-8460-41818f9d6fad None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:20.056915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:20.079506 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f62a6-c123-468f-8460-41818f9d6fad None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:20.114207 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd05a5f-bf0b-478c-9a16-79ec7658a86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:20.114870 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd05a5f-bf0b-478c-9a16-79ec7658a86f None admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:20.115156 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd05a5f-bf0b-478c-9a16-79ec7658a86f None admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:20.115438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd05a5f-bf0b-478c-9a16-79ec7658a86f None admin] PATH_INFO: `/v3/services` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:20.116144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd05a5f-bf0b-478c-9a16-79ec7658a86f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80419) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:55:20.116483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd05a5f-bf0b-478c-9a16-79ec7658a86f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:20.118622 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd05a5f-bf0b-478c-9a16-79ec7658a86f None admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:20.123764 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 654/1305] 10.222.0.88 () {56 vars in 1116 bytes} [Thu Jun 16 14:55:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:20.129692 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afe69b-0295-4a92-bcf4-de547310d536 None admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:20.237620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afe69b-0295-4a92-bcf4-de547310d536 None admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:20.277243 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ef081-f2ba-439d-95db-ba2bc31a3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:20.277915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ef081-f2ba-439d-95db-ba2bc31a3657 None admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:20.278207 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ef081-f2ba-439d-95db-ba2bc31a3657 None admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:20.278485 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ef081-f2ba-439d-95db-ba2bc31a3657 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:20.279257 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ef081-f2ba-439d-95db-ba2bc31a3657 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:20.280806 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ef081-f2ba-439d-95db-ba2bc31a3657 None admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:20.478128 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 652/1306] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 14:55:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:55:30.142278 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=30 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:30.472586 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:44.846764 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e5320-b2e8-4819-9099-aac87a713d1f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:44.848545 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e5320-b2e8-4819-9099-aac87a713d1f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:44.849012 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e5320-b2e8-4819-9099-aac87a713d1f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:44.854576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 655/1307] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:03:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:03:44.856202 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a685d7b-efde-4888-ad89-b12066aa382d None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:44.860481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a685d7b-efde-4888-ad89-b12066aa382d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:44.860481 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a685d7b-efde-4888-ad89-b12066aa382d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:45.031999 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8a685d7b-efde-4888-ad89-b12066aa382d None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:03:45.034199 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8a685d7b-efde-4888-ad89-b12066aa382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:45.261719 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 653/1308] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:03:45.317692 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08c467-161c-4d08-b793-93bd034a4b6a None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:45.318619 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08c467-161c-4d08-b793-93bd034a4b6a None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:45.319085 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08c467-161c-4d08-b793-93bd034a4b6a None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:45.320932 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 656/1309] 10.222.0.88 () {54 vars in 964 bytes} [Thu Jun 16 15:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:03:45.327119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffccc33-3e2c-4a90-8936-b34c37a4767c None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:45.327537 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffccc33-3e2c-4a90-8936-b34c37a4767c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:45.327886 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffccc33-3e2c-4a90-8936-b34c37a4767c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:45.415975 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-7ffccc33-3e2c-4a90-8936-b34c37a4767c None None] MFA Rules not processed for user `58a2b345fe914d2cb97756e6fdf30010`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:03:45.417258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-7ffccc33-3e2c-4a90-8936-b34c37a4767c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:45.443546 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:45.510161 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 654/1310] 10.222.0.88 () {58 vars in 1061 bytes} [Thu Jun 16 15:03:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:03:45.521287 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08c467-161c-4d08-b793-93bd034a4b6a None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:03:45.523930 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ac08c467-161c-4d08-b793-93bd034a4b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:45.650525 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08c467-161c-4d08-b793-93bd034a4b6a None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:03:45.739966 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd084ac-8d6a-42f8-b086-e72b511722f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:03:45.740921 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd084ac-8d6a-42f8-b086-e72b511722f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:45.741364 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd084ac-8d6a-42f8-b086-e72b511722f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:45.741827 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd084ac-8d6a-42f8-b086-e72b511722f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:45.744033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fdd084ac-8d6a-42f8-b086-e72b511722f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:45.824342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:45.892647 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd084ac-8d6a-42f8-b086-e72b511722f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:03:45.894880 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd084ac-8d6a-42f8-b086-e72b511722f6 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:03:45.917375 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 657/1311] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:03:48.629715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0de47-5c5f-4c45-9738-95cb77ee2284 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:48.630573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0de47-5c5f-4c45-9738-95cb77ee2284 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:48.630573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0de47-5c5f-4c45-9738-95cb77ee2284 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:48.635694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 655/1312] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:03:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:03:48.640826 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7ce9-6807-4303-b627-3a4c52fd6fc1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:48.641317 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7ce9-6807-4303-b627-3a4c52fd6fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:48.641412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7ce9-6807-4303-b627-3a4c52fd6fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:48.723987 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-df1d7ce9-6807-4303-b627-3a4c52fd6fc1 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:03:48.725612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-df1d7ce9-6807-4303-b627-3a4c52fd6fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:48.849712 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 658/1313] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:03:48.904455 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcd5ce-3194-474f-8ae1-1393fbc6281e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:48.904819 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcd5ce-3194-474f-8ae1-1393fbc6281e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:48.905114 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcd5ce-3194-474f-8ae1-1393fbc6281e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:48.906528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 656/1314] 10.222.0.88 () {54 vars in 964 bytes} [Thu Jun 16 15:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:03:48.914045 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ced71-ae80-4609-8fb4-5530d44f8eb2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:48.914335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ced71-ae80-4609-8fb4-5530d44f8eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:48.914643 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ced71-ae80-4609-8fb4-5530d44f8eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:48.927631 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:49.015744 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fc5ced71-ae80-4609-8fb4-5530d44f8eb2 None None] MFA Rules not processed for user `58a2b345fe914d2cb97756e6fdf30010`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:03:49.017138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fc5ced71-ae80-4609-8fb4-5530d44f8eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:49.101471 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 659/1315] 10.222.0.88 () {58 vars in 1061 bytes} [Thu Jun 16 15:03:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:03:49.110931 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcd5ce-3194-474f-8ae1-1393fbc6281e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:03:49.112605 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-cddcd5ce-3194-474f-8ae1-1393fbc6281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:49.218708 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcd5ce-3194-474f-8ae1-1393fbc6281e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:03:49.293770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:49.296344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350c742-eca6-4ae1-8909-9e5d09b748d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:03:49.296997 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350c742-eca6-4ae1-8909-9e5d09b748d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:49.297283 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350c742-eca6-4ae1-8909-9e5d09b748d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:49.297593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350c742-eca6-4ae1-8909-9e5d09b748d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:49.299308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3350c742-eca6-4ae1-8909-9e5d09b748d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:49.409718 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350c742-eca6-4ae1-8909-9e5d09b748d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:03:49.411495 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350c742-eca6-4ae1-8909-9e5d09b748d2 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:03:49.431013 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 657/1316] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:03:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:03:52.375719 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b826a-de46-4fbf-a02a-026e6233d82b None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:52.376294 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b826a-de46-4fbf-a02a-026e6233d82b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:52.376294 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b826a-de46-4fbf-a02a-026e6233d82b None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:52.377686 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 660/1317] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:03:52.383953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b8b67-683c-4f71-99ae-515c6e3828a6 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:52.384398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b8b67-683c-4f71-99ae-515c6e3828a6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:52.384550 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b8b67-683c-4f71-99ae-515c6e3828a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:52.448294 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b03b8b67-683c-4f71-99ae-515c6e3828a6 None None] MFA Rules not processed for user `d0a54582166343e5baada31da5b8f7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:03:52.449511 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b03b8b67-683c-4f71-99ae-515c6e3828a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:52.501433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:52.548063 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 658/1318] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:03:52 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:03:52.587732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b826a-de46-4fbf-a02a-026e6233d82b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:03:52.589328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9d4b826a-de46-4fbf-a02a-026e6233d82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:52.693142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b826a-de46-4fbf-a02a-026e6233d82b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:03:52.695852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:52.769497 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295abc3-484b-4bc0-b015-fbb3633295ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:03:52.770161 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295abc3-484b-4bc0-b015-fbb3633295ba service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:52.770489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295abc3-484b-4bc0-b015-fbb3633295ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:52.770810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295abc3-484b-4bc0-b015-fbb3633295ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:52.772564 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-3295abc3-484b-4bc0-b015-fbb3633295ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:52.884819 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295abc3-484b-4bc0-b015-fbb3633295ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:03:52.886275 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295abc3-484b-4bc0-b015-fbb3633295ba service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:03:52.905255 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 661/1319] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:03:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:03:55.857753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9414d8e-0736-4b59-a83e-4a3848d7f725 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:55.858305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9414d8e-0736-4b59-a83e-4a3848d7f725 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:55.858305 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9414d8e-0736-4b59-a83e-4a3848d7f725 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:55.859694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 659/1320] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:03:55.865023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b3c45-277f-4b6f-90e2-ea41a5cd7eb2 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:55.865433 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b3c45-277f-4b6f-90e2-ea41a5cd7eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:55.865593 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b3c45-277f-4b6f-90e2-ea41a5cd7eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:55.905132 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:55.930422 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0c5b3c45-277f-4b6f-90e2-ea41a5cd7eb2 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:03:55.931705 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0c5b3c45-277f-4b6f-90e2-ea41a5cd7eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:56.031586 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 662/1321] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:03:56.070753 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9414d8e-0736-4b59-a83e-4a3848d7f725 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:03:56.072313 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e9414d8e-0736-4b59-a83e-4a3848d7f725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:56.177110 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9414d8e-0736-4b59-a83e-4a3848d7f725 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:03:56.250098 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29837af6-5724-458a-b91a-5866a50edff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:03:56.250743 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29837af6-5724-458a-b91a-5866a50edff1 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:56.251036 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29837af6-5724-458a-b91a-5866a50edff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:56.251410 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29837af6-5724-458a-b91a-5866a50edff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:56.253795 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-29837af6-5724-458a-b91a-5866a50edff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:56.261584 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:56.364356 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29837af6-5724-458a-b91a-5866a50edff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:03:56.365862 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29837af6-5724-458a-b91a-5866a50edff1 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:03:56.384987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 660/1322] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:03:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:03:59.565028 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f96dbc-1b15-48dc-baae-6beb34572301 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:59.565572 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f96dbc-1b15-48dc-baae-6beb34572301 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:59.565856 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f96dbc-1b15-48dc-baae-6beb34572301 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:59.567266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 663/1323] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:03:59.572698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262f417-6246-4d98-8496-7203e24e4929 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:59.573130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262f417-6246-4d98-8496-7203e24e4929 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:59.573419 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262f417-6246-4d98-8496-7203e24e4929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:59.640279 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b262f417-6246-4d98-8496-7203e24e4929 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:03:59.641590 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b262f417-6246-4d98-8496-7203e24e4929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:59.711321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:59.756046 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 661/1324] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:03:59.793527 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f96dbc-1b15-48dc-baae-6beb34572301 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:03:59.795227 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-97f96dbc-1b15-48dc-baae-6beb34572301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:59.905140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f96dbc-1b15-48dc-baae-6beb34572301 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:03:59.918052 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:59.980543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20080c-ad8c-4fea-ad67-65a5f5049f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:03:59.981196 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20080c-ad8c-4fea-ad67-65a5f5049f77 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:59.981483 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20080c-ad8c-4fea-ad67-65a5f5049f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:59.981762 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20080c-ad8c-4fea-ad67-65a5f5049f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:59.983510 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5a20080c-ad8c-4fea-ad67-65a5f5049f77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:00.099685 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20080c-ad8c-4fea-ad67-65a5f5049f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:00.101212 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20080c-ad8c-4fea-ad67-65a5f5049f77 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:00.120778 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 664/1325] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:03:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:02.452144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428442-ea56-4973-85c0-54cc50f4cc6e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:02.452705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428442-ea56-4973-85c0-54cc50f4cc6e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:02.452705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95428442-ea56-4973-85c0-54cc50f4cc6e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:02.454163 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 662/1326] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:02.459982 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2841c7-74a2-48b5-9d4f-c9815a3df1f9 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:02.460420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2841c7-74a2-48b5-9d4f-c9815a3df1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:02.460589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2841c7-74a2-48b5-9d4f-c9815a3df1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:02.513721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:02.531935 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-7a2841c7-74a2-48b5-9d4f-c9815a3df1f9 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:02.533163 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7a2841c7-74a2-48b5-9d4f-c9815a3df1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:02.640438 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 665/1327] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:02.681256 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428442-ea56-4973-85c0-54cc50f4cc6e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:02.682801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-95428442-ea56-4973-85c0-54cc50f4cc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:02.790558 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95428442-ea56-4973-85c0-54cc50f4cc6e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:02.869334 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ff13-e3f3-49d6-ba8a-d7be1a7f5c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:02.869986 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ff13-e3f3-49d6-ba8a-d7be1a7f5c52 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:02.870280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ff13-e3f3-49d6-ba8a-d7be1a7f5c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:02.870563 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ff13-e3f3-49d6-ba8a-d7be1a7f5c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:02.872201 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b790ff13-e3f3-49d6-ba8a-d7be1a7f5c52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:02.889511 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:02.984747 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790ff13-e3f3-49d6-ba8a-d7be1a7f5c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:02.986367 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790ff13-e3f3-49d6-ba8a-d7be1a7f5c52 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:03.010048 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 663/1328] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:05.269755 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626f8d7-f64d-460c-b0de-8627e0db45f4 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:05.270291 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626f8d7-f64d-460c-b0de-8627e0db45f4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:05.270389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626f8d7-f64d-460c-b0de-8627e0db45f4 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:05.271978 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 666/1329] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:05.277049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439d5c0-8a43-453b-a80b-ec4d2ba6d948 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:05.277508 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439d5c0-8a43-453b-a80b-ec4d2ba6d948 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:05.277652 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439d5c0-8a43-453b-a80b-ec4d2ba6d948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:05.343456 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8439d5c0-8a43-453b-a80b-ec4d2ba6d948 None None] MFA Rules not processed for user `d0a54582166343e5baada31da5b8f7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:05.344694 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8439d5c0-8a43-453b-a80b-ec4d2ba6d948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:05.411609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:05.446603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 664/1330] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:04:05.484220 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626f8d7-f64d-460c-b0de-8627e0db45f4 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:05.485952 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d626f8d7-f64d-460c-b0de-8627e0db45f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:05.614991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626f8d7-f64d-460c-b0de-8627e0db45f4 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:05.636740 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:05.695084 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715c520-d588-4067-bfff-46d27898969f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:05.695944 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715c520-d588-4067-bfff-46d27898969f service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:05.696351 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715c520-d588-4067-bfff-46d27898969f service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:05.696747 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715c520-d588-4067-bfff-46d27898969f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:05.698445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e715c520-d588-4067-bfff-46d27898969f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:05.810400 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715c520-d588-4067-bfff-46d27898969f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:05.811839 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715c520-d588-4067-bfff-46d27898969f service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:05.831325 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 667/1331] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:05 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:08.652320 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22abd3-9c2a-4d6e-b8f5-0c2b460899cd None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:08.652856 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22abd3-9c2a-4d6e-b8f5-0c2b460899cd None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:08.652856 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22abd3-9c2a-4d6e-b8f5-0c2b460899cd None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:08.654277 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 665/1332] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:08.659371 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661f498-b3b2-48fa-a72c-ee4fb896cd93 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:08.659797 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661f498-b3b2-48fa-a72c-ee4fb896cd93 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:08.660012 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661f498-b3b2-48fa-a72c-ee4fb896cd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:08.722137 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:08.724993 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6661f498-b3b2-48fa-a72c-ee4fb896cd93 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:08.726198 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6661f498-b3b2-48fa-a72c-ee4fb896cd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:08.822735 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 668/1333] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:08.858428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22abd3-9c2a-4d6e-b8f5-0c2b460899cd None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:08.859953 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5e22abd3-9c2a-4d6e-b8f5-0c2b460899cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:08.963299 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22abd3-9c2a-4d6e-b8f5-0c2b460899cd None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:09.037280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411657dc-e5a4-4400-a8f2-7386c713ac4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:09.037918 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411657dc-e5a4-4400-a8f2-7386c713ac4f service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:09.038200 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411657dc-e5a4-4400-a8f2-7386c713ac4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:09.038477 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411657dc-e5a4-4400-a8f2-7386c713ac4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:09.040113 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-411657dc-e5a4-4400-a8f2-7386c713ac4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:09.067007 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:09.147278 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411657dc-e5a4-4400-a8f2-7386c713ac4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:09.148669 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411657dc-e5a4-4400-a8f2-7386c713ac4f service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:09.166830 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 666/1334] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:11.332252 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938b5df-125a-4c7b-9eb5-869734326290 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:11.332768 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938b5df-125a-4c7b-9eb5-869734326290 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:11.332889 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938b5df-125a-4c7b-9eb5-869734326290 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:11.334276 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 669/1335] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:11.339727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ba77e-d788-4e3b-a13a-de7f1fd631d7 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:11.340152 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ba77e-d788-4e3b-a13a-de7f1fd631d7 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:11.340323 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ba77e-d788-4e3b-a13a-de7f1fd631d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:11.405477 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-dc5ba77e-d788-4e3b-a13a-de7f1fd631d7 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:11.406666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-dc5ba77e-d788-4e3b-a13a-de7f1fd631d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:11.480209 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:11.504505 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 667/1336] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:11.544863 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938b5df-125a-4c7b-9eb5-869734326290 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:11.546324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9938b5df-125a-4c7b-9eb5-869734326290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:11.650462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938b5df-125a-4c7b-9eb5-869734326290 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:11.678834 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:11.725380 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640da59-1425-41b0-a546-08e97cf78f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:11.726023 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640da59-1425-41b0-a546-08e97cf78f1a service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:11.726309 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640da59-1425-41b0-a546-08e97cf78f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:11.726589 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640da59-1425-41b0-a546-08e97cf78f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:11.728271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8640da59-1425-41b0-a546-08e97cf78f1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:11.840054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640da59-1425-41b0-a546-08e97cf78f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:11.841540 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640da59-1425-41b0-a546-08e97cf78f1a service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:11.859864 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 670/1337] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:14.844295 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7752f-aa9c-4277-8e15-ec9690aa05db None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:14.844876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7752f-aa9c-4277-8e15-ec9690aa05db None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:14.844876 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7752f-aa9c-4277-8e15-ec9690aa05db None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:14.846883 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 668/1338] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:14.852013 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924c14-67d6-4e91-aec3-01539936a8e3 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:14.852469 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924c14-67d6-4e91-aec3-01539936a8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:14.852611 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924c14-67d6-4e91-aec3-01539936a8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:14.916798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a8924c14-67d6-4e91-aec3-01539936a8e3 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:14.918026 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a8924c14-67d6-4e91-aec3-01539936a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:14.926914 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:15.015510 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 671/1339] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:15.051915 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7752f-aa9c-4277-8e15-ec9690aa05db None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:15.054919 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-70d7752f-aa9c-4277-8e15-ec9690aa05db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:15.167072 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7752f-aa9c-4277-8e15-ec9690aa05db None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:15.179800 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f36fd-c107-4de0-b05a-b9dedaf8379f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:15.180487 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f36fd-c107-4de0-b05a-b9dedaf8379f service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:15.180776 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f36fd-c107-4de0-b05a-b9dedaf8379f service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:15.181060 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f36fd-c107-4de0-b05a-b9dedaf8379f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:15.182705 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1c9f36fd-c107-4de0-b05a-b9dedaf8379f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:15.290005 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=100 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:15.292202 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f36fd-c107-4de0-b05a-b9dedaf8379f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:15.293664 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f36fd-c107-4de0-b05a-b9dedaf8379f service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:15.314800 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 669/1340] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:19.795410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ba917-834f-424e-bb52-6e508dabef65 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:19.796007 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ba917-834f-424e-bb52-6e508dabef65 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:19.796007 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ba917-834f-424e-bb52-6e508dabef65 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:19.797422 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 672/1341] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:19.803307 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661f3c2-eedb-4e7f-8674-a68048c3304f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:19.803810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661f3c2-eedb-4e7f-8674-a68048c3304f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:19.803920 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661f3c2-eedb-4e7f-8674-a68048c3304f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:19.854258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a661f3c2-eedb-4e7f-8674-a68048c3304f None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:19.855496 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a661f3c2-eedb-4e7f-8674-a68048c3304f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:19.933264 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 670/1342] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:19.972348 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ba917-834f-424e-bb52-6e508dabef65 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:19.974559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-dc6ba917-834f-424e-bb52-6e508dabef65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:19.997693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ba917-834f-424e-bb52-6e508dabef65 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:20.001284 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88391cb4-9b7c-4709-8f38-2bbd9301b8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:20.001985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88391cb4-9b7c-4709-8f38-2bbd9301b8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:20.002272 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88391cb4-9b7c-4709-8f38-2bbd9301b8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:20.002552 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88391cb4-9b7c-4709-8f38-2bbd9301b8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:20.012050 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88391cb4-9b7c-4709-8f38-2bbd9301b8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:20.013524 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88391cb4-9b7c-4709-8f38-2bbd9301b8d1 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:20.023231 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 673/1343] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:04:23.176270 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292d83a-77e8-4417-86f6-fcb583a278c8 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:23.176853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292d83a-77e8-4417-86f6-fcb583a278c8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:23.176853 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292d83a-77e8-4417-86f6-fcb583a278c8 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:23.178234 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 671/1344] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:23.183963 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f36d-d992-4e29-8744-866ff9752a39 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:23.184574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f36d-d992-4e29-8744-866ff9752a39 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:23.184718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421f36d-d992-4e29-8744-866ff9752a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:23.218781 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-1421f36d-d992-4e29-8744-866ff9752a39 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:23.220086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-1421f36d-d992-4e29-8744-866ff9752a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:23.228367 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 674/1345] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:23.267483 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292d83a-77e8-4417-86f6-fcb583a278c8 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:23.278408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292d83a-77e8-4417-86f6-fcb583a278c8 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:23.282132 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e86714-3ac1-44e2-a507-e705cea160ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:23.282761 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e86714-3ac1-44e2-a507-e705cea160ad service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:23.283049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e86714-3ac1-44e2-a507-e705cea160ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:23.283337 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e86714-3ac1-44e2-a507-e705cea160ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:23.292430 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e86714-3ac1-44e2-a507-e705cea160ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:23.293869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e86714-3ac1-44e2-a507-e705cea160ad service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:23.303502 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 672/1346] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:27.695010 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63b9d5-1242-460a-b050-4453e7aef4eb None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:27.695571 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63b9d5-1242-460a-b050-4453e7aef4eb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:27.695571 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63b9d5-1242-460a-b050-4453e7aef4eb None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:27.697124 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 675/1347] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:27.702299 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a7d4e-901b-4b95-bfc7-7ebfa896fa28 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:27.702742 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a7d4e-901b-4b95-bfc7-7ebfa896fa28 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:27.702861 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a7d4e-901b-4b95-bfc7-7ebfa896fa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:27.722052 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-874a7d4e-901b-4b95-bfc7-7ebfa896fa28 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:27.723299 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-874a7d4e-901b-4b95-bfc7-7ebfa896fa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:27.731653 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 673/1348] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:27.770439 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63b9d5-1242-460a-b050-4453e7aef4eb None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:27.772280 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-7d63b9d5-1242-460a-b050-4453e7aef4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:27.785534 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63b9d5-1242-460a-b050-4453e7aef4eb None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:27.789164 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd373d5-f35c-4d4d-a4fc-354e9d2b1f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:27.790850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd373d5-f35c-4d4d-a4fc-354e9d2b1f85 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:27.790850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd373d5-f35c-4d4d-a4fc-354e9d2b1f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:27.790850 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd373d5-f35c-4d4d-a4fc-354e9d2b1f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:27.800328 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd373d5-f35c-4d4d-a4fc-354e9d2b1f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:27.801765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd373d5-f35c-4d4d-a4fc-354e9d2b1f85 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:27.811351 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 676/1349] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:33.060180 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2975-2d57-4b8f-bf06-9a94c6e9bfb2 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:33.061035 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2975-2d57-4b8f-bf06-9a94c6e9bfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:33.061035 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e2975-2d57-4b8f-bf06-9a94c6e9bfb2 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:33.062211 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 674/1350] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 15:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:33.067842 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a957c-3fa8-45ef-9077-df86acf8f4af None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:33.068449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a957c-3fa8-45ef-9077-df86acf8f4af None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:33.068554 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a957c-3fa8-45ef-9077-df86acf8f4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:33.147676 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-037a957c-3fa8-45ef-9077-df86acf8f4af None None] MFA Rules not processed for user `8824338846a940f0865b616ec42cce61`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:33.149393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-037a957c-3fa8-45ef-9077-df86acf8f4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:33.192120 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:33.236200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 677/1351] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:37.501419 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52ad13-e59f-4877-8d82-8d0006fc9c7c None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:37.502088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52ad13-e59f-4877-8d82-8d0006fc9c7c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:37.502088 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52ad13-e59f-4877-8d82-8d0006fc9c7c None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:37.503398 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 675/1352] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 15:04:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:37.509580 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d3c27-fa91-4304-8f9e-f5fb8197fa02 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:37.510111 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d3c27-fa91-4304-8f9e-f5fb8197fa02 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:37.510227 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d3c27-fa91-4304-8f9e-f5fb8197fa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:37.529793 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e40d3c27-fa91-4304-8f9e-f5fb8197fa02 None None] MFA Rules not processed for user `8824338846a940f0865b616ec42cce61`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:37.531002 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e40d3c27-fa91-4304-8f9e-f5fb8197fa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:37.539988 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 678/1353] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:04:37.716371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=60 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:39.035744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11d692-884f-4d5e-a803-ad556e56b4a3 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:39.036426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11d692-884f-4d5e-a803-ad556e56b4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:39.036426 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11d692-884f-4d5e-a803-ad556e56b4a3 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:39.037867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 676/1354] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 15:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:39.042889 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965085cd-a09c-420d-a65a-96e0542034ad None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:39.043339 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965085cd-a09c-420d-a65a-96e0542034ad None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:39.043519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965085cd-a09c-420d-a65a-96e0542034ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:39.063571 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-965085cd-a09c-420d-a65a-96e0542034ad None None] MFA Rules not processed for user `8824338846a940f0865b616ec42cce61`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:39.065171 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-965085cd-a09c-420d-a65a-96e0542034ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:39.075653 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 679/1355] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:04:42.841267 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492b82b-7411-47f3-a2b4-4c2bcd7c83eb None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:42.841813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492b82b-7411-47f3-a2b4-4c2bcd7c83eb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:42.841938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492b82b-7411-47f3-a2b4-4c2bcd7c83eb None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:42.843362 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 677/1356] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:42.848861 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e23ea-964e-469d-8e20-671b5f242ca5 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:42.849367 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e23ea-964e-469d-8e20-671b5f242ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:42.849484 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e23ea-964e-469d-8e20-671b5f242ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:42.870604 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d38e23ea-964e-469d-8e20-671b5f242ca5 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:42.871891 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d38e23ea-964e-469d-8e20-671b5f242ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:42.881318 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 680/1357] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:42.939741 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba81d-96b9-4767-9996-21f18bfebf5f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:42.940325 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba81d-96b9-4767-9996-21f18bfebf5f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:42.941040 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba81d-96b9-4767-9996-21f18bfebf5f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:42.943246 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 678/1358] 10.222.0.88 () {54 vars in 961 bytes} [Thu Jun 16 15:04:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:42.952081 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83e11a-0630-4d02-bad5-5f074d143408 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:42.952700 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83e11a-0630-4d02-bad5-5f074d143408 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:42.953238 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83e11a-0630-4d02-bad5-5f074d143408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:43.007227 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2e83e11a-0630-4d02-bad5-5f074d143408 None None] MFA Rules not processed for user `5368284e42d84e77a6fdb88c2e8dcd3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:43.008562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2e83e11a-0630-4d02-bad5-5f074d143408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:43.100473 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 681/1359] 10.222.0.88 () {58 vars in 1058 bytes} [Thu Jun 16 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:04:43.105893 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba81d-96b9-4767-9996-21f18bfebf5f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:43.121121 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba81d-96b9-4767-9996-21f18bfebf5f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:43.127380 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9401b5a-08ee-400e-87c7-3f96e10800f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:43.128016 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9401b5a-08ee-400e-87c7-3f96e10800f3 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:43.128330 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9401b5a-08ee-400e-87c7-3f96e10800f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:43.128609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9401b5a-08ee-400e-87c7-3f96e10800f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:43.137428 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9401b5a-08ee-400e-87c7-3f96e10800f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:43.139014 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9401b5a-08ee-400e-87c7-3f96e10800f3 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:43.148987 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 679/1360] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:45.531267 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f764acd-772f-46dc-b57a-1b0b962ee57d None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:45.531804 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f764acd-772f-46dc-b57a-1b0b962ee57d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:45.531804 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f764acd-772f-46dc-b57a-1b0b962ee57d None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:45.533425 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 682/1361] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:45.538395 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7084a2-be91-4636-bfb7-a481ce7a6943 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:45.538827 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7084a2-be91-4636-bfb7-a481ce7a6943 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:45.538957 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7084a2-be91-4636-bfb7-a481ce7a6943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:45.557223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ba7084a2-be91-4636-bfb7-a481ce7a6943 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:45.558679 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ba7084a2-be91-4636-bfb7-a481ce7a6943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:45.567253 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 680/1362] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:04:45.613897 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f764acd-772f-46dc-b57a-1b0b962ee57d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:45.633322 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f764acd-772f-46dc-b57a-1b0b962ee57d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:45.639794 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b902d-53ba-47ba-8bdc-c203d385d83b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:45.641559 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b902d-53ba-47ba-8bdc-c203d385d83b service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:45.642110 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b902d-53ba-47ba-8bdc-c203d385d83b service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:45.642401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b902d-53ba-47ba-8bdc-c203d385d83b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:45.652530 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b902d-53ba-47ba-8bdc-c203d385d83b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:45.654200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b902d-53ba-47ba-8bdc-c203d385d83b service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:45.665743 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 683/1363] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:48.363608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478a16-1d3e-4aca-b073-ef41e8fb140f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:48.364383 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478a16-1d3e-4aca-b073-ef41e8fb140f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:48.364383 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478a16-1d3e-4aca-b073-ef41e8fb140f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:48.367240 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 681/1364] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:48.372669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bd6c9-ba31-442e-842f-dd402004ef41 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:48.375410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bd6c9-ba31-442e-842f-dd402004ef41 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:48.375410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bd6c9-ba31-442e-842f-dd402004ef41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:48.393421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-a13bd6c9-ba31-442e-842f-dd402004ef41 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:48.394620 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-a13bd6c9-ba31-442e-842f-dd402004ef41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:48.403419 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 684/1365] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:48.455396 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4af38a-198b-4a7e-88b4-8c67d896cc36 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:48.455874 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4af38a-198b-4a7e-88b4-8c67d896cc36 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:48.456394 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4af38a-198b-4a7e-88b4-8c67d896cc36 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:48.460710 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 682/1366] 10.222.0.88 () {54 vars in 961 bytes} [Thu Jun 16 15:04:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:04:48.466899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b871c-3c04-414e-88bc-d8a4397b1077 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:48.467293 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b871c-3c04-414e-88bc-d8a4397b1077 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:48.467659 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b871c-3c04-414e-88bc-d8a4397b1077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:48.487311 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-d15b871c-3c04-414e-88bc-d8a4397b1077 None None] MFA Rules not processed for user `5368284e42d84e77a6fdb88c2e8dcd3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:48.488551 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-d15b871c-3c04-414e-88bc-d8a4397b1077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:48.509197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 685/1367] 10.222.0.88 () {58 vars in 1058 bytes} [Thu Jun 16 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:48.520341 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4af38a-198b-4a7e-88b4-8c67d896cc36 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:48.536361 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4af38a-198b-4a7e-88b4-8c67d896cc36 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:48.543734 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c96a6a-37c6-4ee0-9b1c-21caa2336600 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:48.543734 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c96a6a-37c6-4ee0-9b1c-21caa2336600 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:48.543734 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c96a6a-37c6-4ee0-9b1c-21caa2336600 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:48.544087 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c96a6a-37c6-4ee0-9b1c-21caa2336600 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:48.564786 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c96a6a-37c6-4ee0-9b1c-21caa2336600 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:48.567586 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c96a6a-37c6-4ee0-9b1c-21caa2336600 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:48.581128 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 683/1368] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:51.028458 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b0555-5d67-4afb-90d4-8ddab950602c None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:51.029043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b0555-5d67-4afb-90d4-8ddab950602c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:51.029043 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b0555-5d67-4afb-90d4-8ddab950602c None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:51.030393 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 686/1369] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:04:51.036617 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b02c54-f566-4c62-9f0a-4c96dd9a85a4 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:51.037141 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b02c54-f566-4c62-9f0a-4c96dd9a85a4 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:51.037335 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b02c54-f566-4c62-9f0a-4c96dd9a85a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:51.055911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-f6b02c54-f566-4c62-9f0a-4c96dd9a85a4 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:51.057149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-f6b02c54-f566-4c62-9f0a-4c96dd9a85a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:51.065217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 684/1370] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:51.109776 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b0555-5d67-4afb-90d4-8ddab950602c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:51.123912 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b0555-5d67-4afb-90d4-8ddab950602c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:51.129698 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf7362-adb8-49db-b258-d1880cc03675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:51.130342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf7362-adb8-49db-b258-d1880cc03675 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:51.130632 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf7362-adb8-49db-b258-d1880cc03675 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:51.130911 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf7362-adb8-49db-b258-d1880cc03675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:51.141333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf7362-adb8-49db-b258-d1880cc03675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:51.143343 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf7362-adb8-49db-b258-d1880cc03675 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:51.155582 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 687/1371] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:51.453340 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7928b0a-5ed8-4100-bcb6-9a5316163654 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:51.454121 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7928b0a-5ed8-4100-bcb6-9a5316163654 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:51.454121 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7928b0a-5ed8-4100-bcb6-9a5316163654 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:51.455535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 685/1372] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:51.461182 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5c5bb-b5a1-474d-8e33-f4e3870666a3 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:51.461481 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5c5bb-b5a1-474d-8e33-f4e3870666a3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:51.461949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5c5bb-b5a1-474d-8e33-f4e3870666a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:51.482242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-95c5c5bb-b5a1-474d-8e33-f4e3870666a3 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:51.483843 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-95c5c5bb-b5a1-474d-8e33-f4e3870666a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:51.496273 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 688/1373] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:51.802892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6feba-d1f0-4963-9e0a-0902d2a4ada5 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:51.802892 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6feba-d1f0-4963-9e0a-0902d2a4ada5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:51.804568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6feba-d1f0-4963-9e0a-0902d2a4ada5 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:51.805450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 686/1374] 10.222.0.88 () {54 vars in 963 bytes} [Thu Jun 16 15:04:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:51.810567 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bcdd0-3b41-4253-b5ae-450d376eaad6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:51.810977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bcdd0-3b41-4253-b5ae-450d376eaad6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:51.811364 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bcdd0-3b41-4253-b5ae-450d376eaad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:51.839861 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:51.865042 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-fb9bcdd0-3b41-4253-b5ae-450d376eaad6 None None] MFA Rules not processed for user `b39d53d1def347b6b36067e3e93aeb98`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:51.866517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-fb9bcdd0-3b41-4253-b5ae-450d376eaad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:51.931233 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 689/1375] 10.222.0.88 () {58 vars in 1060 bytes} [Thu Jun 16 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:51.940491 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6feba-d1f0-4963-9e0a-0902d2a4ada5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:51.953081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a6feba-d1f0-4963-9e0a-0902d2a4ada5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:51.956992 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dc4a9-4652-452b-bf57-7bfcd604f78f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b39d53d1def347b6b36067e3e93aeb98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:51.957770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dc4a9-4652-452b-bf57-7bfcd604f78f service cinder] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:51.958159 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dc4a9-4652-452b-bf57-7bfcd604f78f service cinder] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:51.958539 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dc4a9-4652-452b-bf57-7bfcd604f78f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:51.969459 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409dc4a9-4652-452b-bf57-7bfcd604f78f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:51.971009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409dc4a9-4652-452b-bf57-7bfcd604f78f service cinder] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:51.981873 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 687/1376] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:53.465759 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc558da-b7b5-4599-a061-3a2caddbabec None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:53.466353 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc558da-b7b5-4599-a061-3a2caddbabec None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:53.466512 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc558da-b7b5-4599-a061-3a2caddbabec None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:53.467885 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 690/1377] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:53.473952 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeddeb-765f-4404-adeb-ee93d0d26437 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:53.474452 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeddeb-765f-4404-adeb-ee93d0d26437 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:53.474552 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeddeb-765f-4404-adeb-ee93d0d26437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:53.492731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8dfeddeb-765f-4404-adeb-ee93d0d26437 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:53.494359 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8dfeddeb-765f-4404-adeb-ee93d0d26437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:53.502793 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 688/1378] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:53.548588 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc558da-b7b5-4599-a061-3a2caddbabec None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:53.563494 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc558da-b7b5-4599-a061-3a2caddbabec None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:53.569399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3e9d9-bcf8-44e8-bed4-3bdcbd118d7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:53.570080 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3e9d9-bcf8-44e8-bed4-3bdcbd118d7d service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:53.570370 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3e9d9-bcf8-44e8-bed4-3bdcbd118d7d service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:53.570660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3e9d9-bcf8-44e8-bed4-3bdcbd118d7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:53.580610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3e9d9-bcf8-44e8-bed4-3bdcbd118d7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:53.582264 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3e9d9-bcf8-44e8-bed4-3bdcbd118d7d service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:53.595087 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 691/1379] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:04:55.926821 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9163b-bfd3-40fe-a95f-2caea242f86f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:55.927442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9163b-bfd3-40fe-a95f-2caea242f86f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:55.927442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9163b-bfd3-40fe-a95f-2caea242f86f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:55.928808 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 689/1380] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:55.934203 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ab38e-6eb5-4fa6-8022-8d04f61f104b None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:55.934674 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ab38e-6eb5-4fa6-8022-8d04f61f104b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:55.934774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ab38e-6eb5-4fa6-8022-8d04f61f104b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:55.953001 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e04ab38e-6eb5-4fa6-8022-8d04f61f104b None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:55.954416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e04ab38e-6eb5-4fa6-8022-8d04f61f104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:55.962736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 692/1381] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:56.010259 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9163b-bfd3-40fe-a95f-2caea242f86f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:56.026271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9163b-bfd3-40fe-a95f-2caea242f86f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:56.031864 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03c32c-caad-4944-a72d-d1633f9fbccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:56.032563 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03c32c-caad-4944-a72d-d1633f9fbccf service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:56.032855 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03c32c-caad-4944-a72d-d1633f9fbccf service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:56.033141 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03c32c-caad-4944-a72d-d1633f9fbccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:56.041957 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03c32c-caad-4944-a72d-d1633f9fbccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:56.043579 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03c32c-caad-4944-a72d-d1633f9fbccf service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:56.053922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 690/1382] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:58.406099 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d4c29-0948-41d3-9f18-3136af3f700c None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:58.406664 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d4c29-0948-41d3-9f18-3136af3f700c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:58.406761 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d4c29-0948-41d3-9f18-3136af3f700c None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:58.409012 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 693/1383] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:04:58.413533 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec1f2f-c713-4702-9e15-45fecf4fc3bb None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:58.413984 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec1f2f-c713-4702-9e15-45fecf4fc3bb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:58.414110 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec1f2f-c713-4702-9e15-45fecf4fc3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:58.433129 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-a0ec1f2f-c713-4702-9e15-45fecf4fc3bb None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:58.434333 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-a0ec1f2f-c713-4702-9e15-45fecf4fc3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:58.443144 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 691/1384] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:58.486539 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d4c29-0948-41d3-9f18-3136af3f700c None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:58.502468 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d4c29-0948-41d3-9f18-3136af3f700c None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:58.508162 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfd1f0-5821-4512-9535-5067b316f1cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:58.508937 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfd1f0-5821-4512-9535-5067b316f1cd service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:58.509226 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfd1f0-5821-4512-9535-5067b316f1cd service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:58.509713 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfd1f0-5821-4512-9535-5067b316f1cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:58.522816 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfd1f0-5821-4512-9535-5067b316f1cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:58.524432 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfd1f0-5821-4512-9535-5067b316f1cd service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:58.535205 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 694/1385] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:00.735898 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58f00f-83d1-43e7-8e9e-9a1e68048056 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:00.736480 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58f00f-83d1-43e7-8e9e-9a1e68048056 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:00.736635 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58f00f-83d1-43e7-8e9e-9a1e68048056 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:00.738457 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 692/1386] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:00.744155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552b729-9a71-41b9-9ea5-d8fc04cd5ab3 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:00.744614 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552b729-9a71-41b9-9ea5-d8fc04cd5ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:00.744790 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552b729-9a71-41b9-9ea5-d8fc04cd5ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:00.766405 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-e552b729-9a71-41b9-9ea5-d8fc04cd5ab3 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:00.767594 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-e552b729-9a71-41b9-9ea5-d8fc04cd5ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:00.775903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 695/1387] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:00.822837 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58f00f-83d1-43e7-8e9e-9a1e68048056 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:00.837645 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58f00f-83d1-43e7-8e9e-9a1e68048056 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:00.843162 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2650ba-f831-48fa-80ce-accbe406fcfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:00.843794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2650ba-f831-48fa-80ce-accbe406fcfa service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:00.844084 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2650ba-f831-48fa-80ce-accbe406fcfa service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:00.844408 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2650ba-f831-48fa-80ce-accbe406fcfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:00.852676 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2650ba-f831-48fa-80ce-accbe406fcfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:00.854258 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2650ba-f831-48fa-80ce-accbe406fcfa service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:00.864326 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 693/1388] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:03.538384 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665efd6f-3a69-4517-8f46-035183b04dbe None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:03.538916 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665efd6f-3a69-4517-8f46-035183b04dbe None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:03.539027 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665efd6f-3a69-4517-8f46-035183b04dbe None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:03.540480 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 696/1389] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:03.546593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183c65-b473-4a41-ae56-31763ed7716e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:03.547397 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183c65-b473-4a41-ae56-31763ed7716e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:03.547397 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183c65-b473-4a41-ae56-31763ed7716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:03.566351 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3a183c65-b473-4a41-ae56-31763ed7716e None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:03.567713 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3a183c65-b473-4a41-ae56-31763ed7716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:03.575858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 694/1390] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:03.620648 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665efd6f-3a69-4517-8f46-035183b04dbe None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:03.636706 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665efd6f-3a69-4517-8f46-035183b04dbe None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:03.643170 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc272652-c21c-4312-9200-7014f12f80b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:03.643808 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272652-c21c-4312-9200-7014f12f80b5 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:03.644101 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272652-c21c-4312-9200-7014f12f80b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:03.644441 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272652-c21c-4312-9200-7014f12f80b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:03.653041 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc272652-c21c-4312-9200-7014f12f80b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:03.654667 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc272652-c21c-4312-9200-7014f12f80b5 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:03.664219 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 697/1391] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:05:06.244865 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9838bc-294e-430c-8ff4-4692f16acc1e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:06.245434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9838bc-294e-430c-8ff4-4692f16acc1e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:06.245547 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9838bc-294e-430c-8ff4-4692f16acc1e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:06.247055 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 695/1392] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:06.252905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7844d8-ed79-467c-9e9d-dbe8e1d2d088 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:06.253324 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7844d8-ed79-467c-9e9d-dbe8e1d2d088 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:06.253463 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7844d8-ed79-467c-9e9d-dbe8e1d2d088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:06.272412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-6a7844d8-ed79-467c-9e9d-dbe8e1d2d088 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:06.273592 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-6a7844d8-ed79-467c-9e9d-dbe8e1d2d088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:06.282616 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 698/1393] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:06.326304 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9838bc-294e-430c-8ff4-4692f16acc1e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:06.341127 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9838bc-294e-430c-8ff4-4692f16acc1e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:06.347813 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f6521-1fa4-4382-9cd9-c7ed4da3090f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:06.348487 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f6521-1fa4-4382-9cd9-c7ed4da3090f service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:06.348775 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f6521-1fa4-4382-9cd9-c7ed4da3090f service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:06.349054 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f6521-1fa4-4382-9cd9-c7ed4da3090f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:06.357610 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f6521-1fa4-4382-9cd9-c7ed4da3090f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:06.359743 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f6521-1fa4-4382-9cd9-c7ed4da3090f service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:06.369308 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 696/1394] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:08.176245 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e305a-beaa-467f-8215-5dfe0c73c35f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:08.177166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e305a-beaa-467f-8215-5dfe0c73c35f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:08.177460 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e305a-beaa-467f-8215-5dfe0c73c35f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:08.178855 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 699/1395] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:08.184174 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f3685-1112-4559-8882-a2e0d33de0bf None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:08.184662 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f3685-1112-4559-8882-a2e0d33de0bf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:08.184922 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f3685-1112-4559-8882-a2e0d33de0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:08.191274 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=52 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:05:08.203155 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-bb4f3685-1112-4559-8882-a2e0d33de0bf None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:08.207967 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bb4f3685-1112-4559-8882-a2e0d33de0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:08.212466 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 697/1396] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:05:08.268828 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72597de5-1040-474e-989e-673ae2f7e30d None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:08.269121 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72597de5-1040-474e-989e-673ae2f7e30d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:08.269406 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72597de5-1040-474e-989e-673ae2f7e30d None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:08.270769 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 700/1397] 10.222.0.88 () {54 vars in 962 bytes} [Thu Jun 16 15:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:08.275794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e147-7dfd-46e4-9c9f-6098579471cc None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:08.276092 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e147-7dfd-46e4-9c9f-6098579471cc None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:08.276421 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e147-7dfd-46e4-9c9f-6098579471cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:08.325934 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-5fb2e147-7dfd-46e4-9c9f-6098579471cc None None] MFA Rules not processed for user `e7ffcec55ac14f5bb8c3af6be2d4c7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:08.327317 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-5fb2e147-7dfd-46e4-9c9f-6098579471cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:08.388350 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 698/1398] 10.222.0.88 () {58 vars in 1059 bytes} [Thu Jun 16 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:08.396810 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72597de5-1040-474e-989e-673ae2f7e30d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:08.408281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72597de5-1040-474e-989e-673ae2f7e30d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:08.411765 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bfd06-8b4d-4d33-91c9-73d62c783184 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ffcec55ac14f5bb8c3af6be2d4c7c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:08.412443 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bfd06-8b4d-4d33-91c9-73d62c783184 service glance] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:08.412732 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bfd06-8b4d-4d33-91c9-73d62c783184 service glance] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:08.413009 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bfd06-8b4d-4d33-91c9-73d62c783184 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:08.422610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bfd06-8b4d-4d33-91c9-73d62c783184 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:08.424088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bfd06-8b4d-4d33-91c9-73d62c783184 service glance] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:08.433787 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 701/1399] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:08.682056 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f362ee-3bfe-4ba1-8944-dec8f2bd7852 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:08.682416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f362ee-3bfe-4ba1-8944-dec8f2bd7852 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:08.682706 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f362ee-3bfe-4ba1-8944-dec8f2bd7852 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:08.684180 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 699/1400] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:08.689410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7acde-2856-4bca-a4ad-f95ecd2976f1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:08.689770 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7acde-2856-4bca-a4ad-f95ecd2976f1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:08.690114 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7acde-2856-4bca-a4ad-f95ecd2976f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:08.708187 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-19b7acde-2856-4bca-a4ad-f95ecd2976f1 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:08.709499 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-19b7acde-2856-4bca-a4ad-f95ecd2976f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:08.718094 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 702/1401] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:08.763570 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f362ee-3bfe-4ba1-8944-dec8f2bd7852 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:08.779920 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f362ee-3bfe-4ba1-8944-dec8f2bd7852 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:08.785683 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae66f1f-472b-450c-811d-af7bfd791566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:08.786384 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae66f1f-472b-450c-811d-af7bfd791566 service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:08.786672 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae66f1f-472b-450c-811d-af7bfd791566 service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:08.786962 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae66f1f-472b-450c-811d-af7bfd791566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:08.795181 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae66f1f-472b-450c-811d-af7bfd791566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:08.796897 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae66f1f-472b-450c-811d-af7bfd791566 service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:08.807904 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 700/1402] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:09.281985 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55af7-ce46-49b8-8ca9-763d738e9cf0 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:09.282454 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55af7-ce46-49b8-8ca9-763d738e9cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:09.282568 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55af7-ce46-49b8-8ca9-763d738e9cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:09.331584 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9fe55af7-ce46-49b8-8ca9-763d738e9cf0 None None] MFA Rules not processed for user `eadd75d096ac49758fdf012c8d8755ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:09.332859 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9fe55af7-ce46-49b8-8ca9-763d738e9cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:09.377964 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=100 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:05:09.398805 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 703/1403] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:09.445485 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b48081-9efd-4c5d-804e-88638fe3c343 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:09.445951 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b48081-9efd-4c5d-804e-88638fe3c343 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:09.446190 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b48081-9efd-4c5d-804e-88638fe3c343 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:09.447626 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 701/1404] 10.222.0.88 () {54 vars in 942 bytes} [Thu Jun 16 15:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:09.454180 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3376d8-463f-4e74-a392-92c60d4e84ab None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:09.455232 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3376d8-463f-4e74-a392-92c60d4e84ab None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:09.455232 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3376d8-463f-4e74-a392-92c60d4e84ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:09.506168 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ed3376d8-463f-4e74-a392-92c60d4e84ab None None] MFA Rules not processed for user `bf0dd8c9980e4640aafd34946c4193a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:09.507381 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ed3376d8-463f-4e74-a392-92c60d4e84ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:09.581225 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 704/1405] 10.222.0.88 () {58 vars in 1039 bytes} [Thu Jun 16 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:09.590568 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b48081-9efd-4c5d-804e-88638fe3c343 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:09.604434 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b48081-9efd-4c5d-804e-88638fe3c343 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:09.610085 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081e877-bab3-4996-985a-dbfcb2797426 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf0dd8c9980e4640aafd34946c4193a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:09.610724 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081e877-bab3-4996-985a-dbfcb2797426 service swift] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:09.611015 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081e877-bab3-4996-985a-dbfcb2797426 service swift] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:09.611295 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081e877-bab3-4996-985a-dbfcb2797426 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:09.619573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081e877-bab3-4996-985a-dbfcb2797426 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:09.621208 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081e877-bab3-4996-985a-dbfcb2797426 service swift] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:09.630248 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 702/1406] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 15:05:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:05:11.159967 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d7e89-15ef-49cc-bfc8-141b33c9a6ad None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:11.160662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d7e89-15ef-49cc-bfc8-141b33c9a6ad None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:11.160662 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d7e89-15ef-49cc-bfc8-141b33c9a6ad None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:11.162127 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 705/1407] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:11.167198 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9489e-4bd1-4044-80fc-7dd431a80fae None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:11.167656 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9489e-4bd1-4044-80fc-7dd431a80fae None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:11.167769 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9489e-4bd1-4044-80fc-7dd431a80fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:11.185831 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-0bf9489e-4bd1-4044-80fc-7dd431a80fae None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:11.187968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-0bf9489e-4bd1-4044-80fc-7dd431a80fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:11.198433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 703/1408] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:11.245151 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d7e89-15ef-49cc-bfc8-141b33c9a6ad None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:11.260721 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d7e89-15ef-49cc-bfc8-141b33c9a6ad None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:11.268129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580a054-11b7-414a-a922-971a752b124f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:11.268129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580a054-11b7-414a-a922-971a752b124f service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:11.268418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580a054-11b7-414a-a922-971a752b124f service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:11.268508 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580a054-11b7-414a-a922-971a752b124f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:11.278558 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580a054-11b7-414a-a922-971a752b124f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:11.280335 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580a054-11b7-414a-a922-971a752b124f service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:11.290018 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 706/1409] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:21.180587 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=55 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:05:21.285693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=57 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:05:33.901081 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46a52-1fd3-4e88-ac4b-5b0debf48d2f None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:33.901659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46a52-1fd3-4e88-ac4b-5b0debf48d2f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:33.901659 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc46a52-1fd3-4e88-ac4b-5b0debf48d2f None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:33.903023 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 704/1410] 10.222.0.88 () {54 vars in 918 bytes} [Thu Jun 16 15:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:33.908873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c12d-bed3-45fa-9a20-53664ead223d None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:33.909342 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c12d-bed3-45fa-9a20-53664ead223d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:33.909443 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c12d-bed3-45fa-9a20-53664ead223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:33.927142 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-0c87c12d-bed3-45fa-9a20-53664ead223d None None] MFA Rules not processed for user `8824338846a940f0865b616ec42cce61`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:33.928361 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-0c87c12d-bed3-45fa-9a20-53664ead223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:33.937195 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 707/1411] 10.222.0.88 () {58 vars in 1015 bytes} [Thu Jun 16 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:05:33.951960 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a1ba6-5c23-4ef4-b416-362a87c93ff1 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:33.952271 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a1ba6-5c23-4ef4-b416-362a87c93ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:33.952553 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a1ba6-5c23-4ef4-b416-362a87c93ff1 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:33.953891 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 705/1412] 10.222.0.88 () {54 vars in 918 bytes} [Thu Jun 16 15:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:33.958491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca9557-50c4-483f-928b-2ab4668598d6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:33.958788 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca9557-50c4-483f-928b-2ab4668598d6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:33.959078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca9557-50c4-483f-928b-2ab4668598d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:33.977344 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-14ca9557-50c4-483f-928b-2ab4668598d6 None None] MFA Rules not processed for user `5368284e42d84e77a6fdb88c2e8dcd3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:33.978526 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-14ca9557-50c4-483f-928b-2ab4668598d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:33.987685 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 708/1413] 10.222.0.88 () {58 vars in 1015 bytes} [Thu Jun 16 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:36.667344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c04be1-04e2-4f9b-b52e-e7fa8a5fad1e None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:36.667947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c04be1-04e2-4f9b-b52e-e7fa8a5fad1e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:36.667947 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c04be1-04e2-4f9b-b52e-e7fa8a5fad1e None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:36.669365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:36.674913 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289bf1d-cae3-4b0f-aa3d-3cc17625faee None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:36.675338 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289bf1d-cae3-4b0f-aa3d-3cc17625faee None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:36.675487 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289bf1d-cae3-4b0f-aa3d-3cc17625faee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:36.693574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-5289bf1d-cae3-4b0f-aa3d-3cc17625faee None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:36.694776 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-5289bf1d-cae3-4b0f-aa3d-3cc17625faee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:36.703543 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:36.746432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c04be1-04e2-4f9b-b52e-e7fa8a5fad1e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:36.762067 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c04be1-04e2-4f9b-b52e-e7fa8a5fad1e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:36.768665 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528bca0-efbd-4fb4-bf47-eb80798678fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:36.769396 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528bca0-efbd-4fb4-bf47-eb80798678fe service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:36.769744 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528bca0-efbd-4fb4-bf47-eb80798678fe service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:36.770108 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528bca0-efbd-4fb4-bf47-eb80798678fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:36.779367 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528bca0-efbd-4fb4-bf47-eb80798678fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:36.781197 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528bca0-efbd-4fb4-bf47-eb80798678fe service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:36.790823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 707/1416] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:37.748089 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3a08a-b1d5-4b6f-87ea-d8a15e6e126d None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:37.748702 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3a08a-b1d5-4b6f-87ea-d8a15e6e126d None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:37.748702 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3a08a-b1d5-4b6f-87ea-d8a15e6e126d None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:37.750066 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 15:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:37.755321 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44780b20-b7b6-404f-9b55-5e0a4e11615b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:37.755780 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44780b20-b7b6-404f-9b55-5e0a4e11615b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:37.755899 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44780b20-b7b6-404f-9b55-5e0a4e11615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:37.773968 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-44780b20-b7b6-404f-9b55-5e0a4e11615b None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:37.775172 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-44780b20-b7b6-404f-9b55-5e0a4e11615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:37.783009 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:37.938300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3a08a-b1d5-4b6f-87ea-d8a15e6e126d None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:37.952561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3a08a-b1d5-4b6f-87ea-d8a15e6e126d None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:37.957994 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f9e45-51d6-4e9e-90e5-5fd0b130a578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:37.958632 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f9e45-51d6-4e9e-90e5-5fd0b130a578 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:37.958927 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f9e45-51d6-4e9e-90e5-5fd0b130a578 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:37.959216 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f9e45-51d6-4e9e-90e5-5fd0b130a578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:37.967572 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f9e45-51d6-4e9e-90e5-5fd0b130a578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:37.969199 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f9e45-51d6-4e9e-90e5-5fd0b130a578 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:37.978848 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 711/1419] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:38.908737 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef845a77-da1f-47ff-93a1-64f0068d669a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:38.909315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef845a77-da1f-47ff-93a1-64f0068d669a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:38.909315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef845a77-da1f-47ff-93a1-64f0068d669a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:38.910666 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 15:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:38.915622 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71250707-d0bc-4cb2-862b-f2afee8a83a1 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:38.916048 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71250707-d0bc-4cb2-862b-f2afee8a83a1 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:38.916185 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71250707-d0bc-4cb2-862b-f2afee8a83a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:38.935561 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-71250707-d0bc-4cb2-862b-f2afee8a83a1 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:38.936792 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-71250707-d0bc-4cb2-862b-f2afee8a83a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:38.944642 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 15:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:39.100022 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef845a77-da1f-47ff-93a1-64f0068d669a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:39.114875 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef845a77-da1f-47ff-93a1-64f0068d669a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:39.120212 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc229d1c-56f8-47d2-8f9a-1e343dfac99f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:39.120867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229d1c-56f8-47d2-8f9a-1e343dfac99f service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:39.121156 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229d1c-56f8-47d2-8f9a-1e343dfac99f service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:39.121437 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229d1c-56f8-47d2-8f9a-1e343dfac99f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:39.129823 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc229d1c-56f8-47d2-8f9a-1e343dfac99f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:39.131390 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc229d1c-56f8-47d2-8f9a-1e343dfac99f service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:39.141306 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 710/1422] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:40.060155 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b5c72-a4c5-4c24-a4ae-64a2e4af7ff3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:40.060772 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b5c72-a4c5-4c24-a4ae-64a2e4af7ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:40.060772 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b5c72-a4c5-4c24-a4ae-64a2e4af7ff3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:40.062112 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 15:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:40.067346 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd38a1-3751-4122-b3d8-e437c49225ce None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:40.067736 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd38a1-3751-4122-b3d8-e437c49225ce None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:40.067917 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd38a1-3751-4122-b3d8-e437c49225ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:40.086142 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-85fd38a1-3751-4122-b3d8-e437c49225ce None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:40.087344 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-85fd38a1-3751-4122-b3d8-e437c49225ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:40.095770 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 15:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:40.249777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b5c72-a4c5-4c24-a4ae-64a2e4af7ff3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:40.264818 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b5c72-a4c5-4c24-a4ae-64a2e4af7ff3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:40.270100 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381b1f1-e37e-40d7-b124-fa8c6d6cc5dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:40.270854 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381b1f1-e37e-40d7-b124-fa8c6d6cc5dd service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:40.271148 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381b1f1-e37e-40d7-b124-fa8c6d6cc5dd service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:40.271429 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381b1f1-e37e-40d7-b124-fa8c6d6cc5dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:40.279619 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381b1f1-e37e-40d7-b124-fa8c6d6cc5dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:40.281240 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381b1f1-e37e-40d7-b124-fa8c6d6cc5dd service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:40.290520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 714/1425] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:41.207655 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54493db4-d930-401d-a1d3-a4c036123cb0 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:41.208345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54493db4-d930-401d-a1d3-a4c036123cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:41.208345 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54493db4-d930-401d-a1d3-a4c036123cb0 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:41.209611 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 15:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:41.214669 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2dd97-2cc0-46d5-b108-f50bcb1b5fd7 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:41.215104 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2dd97-2cc0-46d5-b108-f50bcb1b5fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:41.215247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2dd97-2cc0-46d5-b108-f50bcb1b5fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:41.233682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-f8a2dd97-2cc0-46d5-b108-f50bcb1b5fd7 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:41.234869 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-f8a2dd97-2cc0-46d5-b108-f50bcb1b5fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:41.242667 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 15:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:41.394634 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54493db4-d930-401d-a1d3-a4c036123cb0 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:41.408888 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54493db4-d930-401d-a1d3-a4c036123cb0 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:41.414203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35496c-2387-41af-a80b-536512ecce3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:41.414831 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35496c-2387-41af-a80b-536512ecce3d service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:41.415120 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35496c-2387-41af-a80b-536512ecce3d service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:41.415402 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35496c-2387-41af-a80b-536512ecce3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:41.424278 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35496c-2387-41af-a80b-536512ecce3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:41.425846 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35496c-2387-41af-a80b-536512ecce3d service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:41.435991 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 713/1428] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:42.380246 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fc303-d8fc-47db-a77d-4dc2d3f4b9bf None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:42.380798 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fc303-d8fc-47db-a77d-4dc2d3f4b9bf None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:42.380899 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fc303-d8fc-47db-a77d-4dc2d3f4b9bf None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:42.382209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 15:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:42.387070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d27663-29ce-4ac3-aeec-88727bb177ba None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:42.388000 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d27663-29ce-4ac3-aeec-88727bb177ba None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:42.388132 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d27663-29ce-4ac3-aeec-88727bb177ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:42.406535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-62d27663-29ce-4ac3-aeec-88727bb177ba None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:42.407780 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-62d27663-29ce-4ac3-aeec-88727bb177ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:42.415672 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 15:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:42.568829 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fc303-d8fc-47db-a77d-4dc2d3f4b9bf None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:42.583569 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fc303-d8fc-47db-a77d-4dc2d3f4b9bf None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:42.588954 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a3454-a0c0-4a88-bbfb-71d53b6f6df1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:42.589585 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a3454-a0c0-4a88-bbfb-71d53b6f6df1 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:42.589871 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a3454-a0c0-4a88-bbfb-71d53b6f6df1 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:42.590150 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a3454-a0c0-4a88-bbfb-71d53b6f6df1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:42.598524 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a3454-a0c0-4a88-bbfb-71d53b6f6df1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:42.600105 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a3454-a0c0-4a88-bbfb-71d53b6f6df1 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:42.609339 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 717/1431] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:43.543727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79eb99-9c17-4268-8756-cdf569344497 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:43.545027 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79eb99-9c17-4268-8756-cdf569344497 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:43.545027 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79eb99-9c17-4268-8756-cdf569344497 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:43.546350 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 15:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:43.551368 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9f7ac-6dee-49ec-b27c-a13098453d51 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:43.551834 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9f7ac-6dee-49ec-b27c-a13098453d51 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:43.551941 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9f7ac-6dee-49ec-b27c-a13098453d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:43.570725 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2cb9f7ac-6dee-49ec-b27c-a13098453d51 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:43.571897 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2cb9f7ac-6dee-49ec-b27c-a13098453d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:43.581635 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:05:43.740579 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79eb99-9c17-4268-8756-cdf569344497 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:43.755867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79eb99-9c17-4268-8756-cdf569344497 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:43.761216 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9815a-bf02-49aa-aa35-ca490c9e9d7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:43.761849 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9815a-bf02-49aa-aa35-ca490c9e9d7a service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:43.762141 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9815a-bf02-49aa-aa35-ca490c9e9d7a service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:43.762469 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9815a-bf02-49aa-aa35-ca490c9e9d7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:43.770573 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9815a-bf02-49aa-aa35-ca490c9e9d7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:43.772151 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9815a-bf02-49aa-aa35-ca490c9e9d7a service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:43.781735 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 716/1434] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:44.712125 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c66698-0944-4332-98bf-08af4c1e02c7 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:44.712715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c66698-0944-4332-98bf-08af4c1e02c7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:44.712715 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c66698-0944-4332-98bf-08af4c1e02c7 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:44.714109 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 15:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:44.720002 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2989d-9002-490c-b69f-0be738a9cd40 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:44.720454 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2989d-9002-490c-b69f-0be738a9cd40 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:44.720607 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2989d-9002-490c-b69f-0be738a9cd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:44.738431 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-8eb2989d-9002-490c-b69f-0be738a9cd40 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:44.739562 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-8eb2989d-9002-490c-b69f-0be738a9cd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:44.747872 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:44.900609 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c66698-0944-4332-98bf-08af4c1e02c7 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:44.915025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c66698-0944-4332-98bf-08af4c1e02c7 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:44.921131 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64407775-80ba-493d-99c0-947420a73177 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:44.921763 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64407775-80ba-493d-99c0-947420a73177 service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:44.922044 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64407775-80ba-493d-99c0-947420a73177 service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:44.922320 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64407775-80ba-493d-99c0-947420a73177 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:44.930562 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64407775-80ba-493d-99c0-947420a73177 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:44.932144 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64407775-80ba-493d-99c0-947420a73177 service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:44.942377 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 720/1437] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:45.908835 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a975724-fc4f-4418-9a80-08fbed634b77 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:45.909443 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a975724-fc4f-4418-9a80-08fbed634b77 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:45.909443 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a975724-fc4f-4418-9a80-08fbed634b77 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:45.910810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 15:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:45.915809 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c22b2-eff3-4160-9660-3b492401d56e None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:45.916300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c22b2-eff3-4160-9660-3b492401d56e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:45.916417 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c22b2-eff3-4160-9660-3b492401d56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:45.934633 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-788c22b2-eff3-4160-9660-3b492401d56e None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:45.936512 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-788c22b2-eff3-4160-9660-3b492401d56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:45.944247 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:46.095938 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a975724-fc4f-4418-9a80-08fbed634b77 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:46.110847 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a975724-fc4f-4418-9a80-08fbed634b77 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:46.116282 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18a778-a7cf-4ab6-9a44-c07a12eb1dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:46.116916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18a778-a7cf-4ab6-9a44-c07a12eb1dec service nova] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:46.117207 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18a778-a7cf-4ab6-9a44-c07a12eb1dec service nova] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:46.117506 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18a778-a7cf-4ab6-9a44-c07a12eb1dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:46.126878 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18a778-a7cf-4ab6-9a44-c07a12eb1dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:46.128756 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18a778-a7cf-4ab6-9a44-c07a12eb1dec service nova] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:46.138203 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 719/1440] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:48.220718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0880f-e4d8-4006-8cf8-9b069dfb108f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:48.221660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0880f-e4d8-4006-8cf8-9b069dfb108f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:48.221660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0880f-e4d8-4006-8cf8-9b069dfb108f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:48.222724 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:48.228171 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c7458-1fe8-4ecf-889b-8289b09c3dcf None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:48.228695 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c7458-1fe8-4ecf-889b-8289b09c3dcf None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:48.228877 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c7458-1fe8-4ecf-889b-8289b09c3dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:48.248861 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-1a6c7458-1fe8-4ecf-889b-8289b09c3dcf None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:48.250101 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-1a6c7458-1fe8-4ecf-889b-8289b09c3dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:48.258620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:48.305615 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0880f-e4d8-4006-8cf8-9b069dfb108f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:48.317984 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0880f-e4d8-4006-8cf8-9b069dfb108f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:48.321574 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19641e39-1310-4faa-b52a-066031de6534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:48.322636 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19641e39-1310-4faa-b52a-066031de6534 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:48.323157 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19641e39-1310-4faa-b52a-066031de6534 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:48.323447 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19641e39-1310-4faa-b52a-066031de6534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:48.337333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19641e39-1310-4faa-b52a-066031de6534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:48.338799 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19641e39-1310-4faa-b52a-066031de6534 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:48.349440 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 723/1443] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:50.695458 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b80651-5e02-4718-9966-2a91b40714ce None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:50.696028 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b80651-5e02-4718-9966-2a91b40714ce None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:50.696028 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b80651-5e02-4718-9966-2a91b40714ce None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:50.697517 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:50.703736 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932fe6d-1a9c-448a-8860-83a5292fc7f4 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:50.704218 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932fe6d-1a9c-448a-8860-83a5292fc7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:50.704357 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932fe6d-1a9c-448a-8860-83a5292fc7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:50.722209 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-9932fe6d-1a9c-448a-8860-83a5292fc7f4 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:50.723399 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9932fe6d-1a9c-448a-8860-83a5292fc7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:50.731160 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:50.771844 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b80651-5e02-4718-9966-2a91b40714ce None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:50.782815 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b80651-5e02-4718-9966-2a91b40714ce None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:50.786414 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fad754-afa4-4847-a909-cb730c9f3f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:50.787047 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fad754-afa4-4847-a909-cb730c9f3f02 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:50.787336 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fad754-afa4-4847-a909-cb730c9f3f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:50.787618 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fad754-afa4-4847-a909-cb730c9f3f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:50.796715 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fad754-afa4-4847-a909-cb730c9f3f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:50.798142 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fad754-afa4-4847-a909-cb730c9f3f02 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:50.807450 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 722/1446] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:53.360852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2270ff-6fc2-49fd-8962-d6417894e787 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:53.361418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2270ff-6fc2-49fd-8962-d6417894e787 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:53.361418 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2270ff-6fc2-49fd-8962-d6417894e787 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:53.362777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:53.368139 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a6858-d4fa-4c1c-9d21-200a4559653f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:53.368624 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a6858-d4fa-4c1c-9d21-200a4559653f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:53.368732 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a6858-d4fa-4c1c-9d21-200a4559653f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:53.388325 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-840a6858-d4fa-4c1c-9d21-200a4559653f None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:53.389620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-840a6858-d4fa-4c1c-9d21-200a4559653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:53.397535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:53.437365 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2270ff-6fc2-49fd-8962-d6417894e787 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:53.447852 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2270ff-6fc2-49fd-8962-d6417894e787 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:53.452033 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736d222-e88f-4658-abe7-ee37f090584e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:53.452692 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736d222-e88f-4658-abe7-ee37f090584e service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:53.452986 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736d222-e88f-4658-abe7-ee37f090584e service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:53.453271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736d222-e88f-4658-abe7-ee37f090584e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:53.462031 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736d222-e88f-4658-abe7-ee37f090584e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:53.463455 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736d222-e88f-4658-abe7-ee37f090584e service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:53.473544 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 726/1449] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:55.804801 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616b285-123e-487d-99f9-e81e72542e5c None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:55.805391 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616b285-123e-487d-99f9-e81e72542e5c None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:55.805391 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616b285-123e-487d-99f9-e81e72542e5c None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:55.806746 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:55.812448 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d0ff0-7d8b-4929-b60d-ba58cb707604 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:55.812894 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d0ff0-7d8b-4929-b60d-ba58cb707604 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:55.813054 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d0ff0-7d8b-4929-b60d-ba58cb707604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:55.831391 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-2d5d0ff0-7d8b-4929-b60d-ba58cb707604 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:55.832618 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-2d5d0ff0-7d8b-4929-b60d-ba58cb707604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:55.840421 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:55.878407 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616b285-123e-487d-99f9-e81e72542e5c None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:55.889743 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616b285-123e-487d-99f9-e81e72542e5c None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:55.893273 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b1497-2677-40a5-a833-c18d44534380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:55.893905 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b1497-2677-40a5-a833-c18d44534380 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:55.894198 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b1497-2677-40a5-a833-c18d44534380 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:55.894484 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b1497-2677-40a5-a833-c18d44534380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:55.903965 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b1497-2677-40a5-a833-c18d44534380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:55.905566 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b1497-2677-40a5-a833-c18d44534380 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:55.914875 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 725/1452] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:58.119811 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e728adf-318d-4cf6-ad35-d4f71908fd62 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:58.120410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e728adf-318d-4cf6-ad35-d4f71908fd62 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:58.120410 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e728adf-318d-4cf6-ad35-d4f71908fd62 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:58.121855 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:58.128175 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e5528-c5f4-4743-bbf2-3cb3279748ab None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:58.128608 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e5528-c5f4-4743-bbf2-3cb3279748ab None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:58.128764 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e5528-c5f4-4743-bbf2-3cb3279748ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:58.146266 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-467e5528-c5f4-4743-bbf2-3cb3279748ab None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:58.147449 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-467e5528-c5f4-4743-bbf2-3cb3279748ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:58.155603 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:58.194307 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e728adf-318d-4cf6-ad35-d4f71908fd62 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:58.205205 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e728adf-318d-4cf6-ad35-d4f71908fd62 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:58.208760 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06aad2-8e46-459b-b0d5-0a41fb0254c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:58.209404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06aad2-8e46-459b-b0d5-0a41fb0254c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:58.209693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06aad2-8e46-459b-b0d5-0a41fb0254c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:58.210004 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06aad2-8e46-459b-b0d5-0a41fb0254c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:58.219560 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06aad2-8e46-459b-b0d5-0a41fb0254c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:58.221057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06aad2-8e46-459b-b0d5-0a41fb0254c4 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:58.230353 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 729/1455] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:00.431914 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9d8cc-5190-4d8c-b5b6-34555b2ac2b6 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:00.432538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9d8cc-5190-4d8c-b5b6-34555b2ac2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:00.432538 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9d8cc-5190-4d8c-b5b6-34555b2ac2b6 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:00.433886 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:00.440051 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9b1c7-778f-4255-81b6-0b71884b4cb6 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:00.442167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9b1c7-778f-4255-81b6-0b71884b4cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:00.442167 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9b1c7-778f-4255-81b6-0b71884b4cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:00.458227 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-c9c9b1c7-778f-4255-81b6-0b71884b4cb6 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:00.459424 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-c9c9b1c7-778f-4255-81b6-0b71884b4cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:00.467918 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:00.506371 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9d8cc-5190-4d8c-b5b6-34555b2ac2b6 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:00.517377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9d8cc-5190-4d8c-b5b6-34555b2ac2b6 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:00.520933 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9ea43-31fa-436e-bcb9-e2527be19738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:00.521567 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9ea43-31fa-436e-bcb9-e2527be19738 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:00.521859 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9ea43-31fa-436e-bcb9-e2527be19738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:00.522140 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9ea43-31fa-436e-bcb9-e2527be19738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:00.531834 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9ea43-31fa-436e-bcb9-e2527be19738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:00.533295 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9ea43-31fa-436e-bcb9-e2527be19738 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:00.542598 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 728/1458] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:02.745092 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768e1d-6234-4acf-a612-8c5166bcdaa5 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:02.745656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768e1d-6234-4acf-a612-8c5166bcdaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:02.745656 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768e1d-6234-4acf-a612-8c5166bcdaa5 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:02.747585 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:02.753027 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84a94c-2e4d-448b-9889-87f6b28a01ac None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:02.753432 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84a94c-2e4d-448b-9889-87f6b28a01ac None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:02.753609 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84a94c-2e4d-448b-9889-87f6b28a01ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:02.759665 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=70 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:02.772555 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-9d84a94c-2e4d-448b-9889-87f6b28a01ac None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:02.773698 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-9d84a94c-2e4d-448b-9889-87f6b28a01ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:02.781542 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:02.818696 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31768e1d-6234-4acf-a612-8c5166bcdaa5 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:02.824313 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=73 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:02.831128 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31768e1d-6234-4acf-a612-8c5166bcdaa5 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:02.834731 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fbd30-3dab-4b0e-ac4d-f11d8df6a226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:02.835501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbd30-3dab-4b0e-ac4d-f11d8df6a226 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:02.835789 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbd30-3dab-4b0e-ac4d-f11d8df6a226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:02.836072 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbd30-3dab-4b0e-ac4d-f11d8df6a226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:02.845138 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fbd30-3dab-4b0e-ac4d-f11d8df6a226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:02.846660 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fbd30-3dab-4b0e-ac4d-f11d8df6a226 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:02.856347 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 732/1461] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:05.043020 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa83f7f-9332-4b80-a95b-c5636d93d37d None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:05.044017 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa83f7f-9332-4b80-a95b-c5636d93d37d None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:05.044017 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa83f7f-9332-4b80-a95b-c5636d93d37d None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:05.045028 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:05.050262 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c549f7-16c5-4de5-9979-48d014e0a68c None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:05.050700 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c549f7-16c5-4de5-9979-48d014e0a68c None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:05.050827 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c549f7-16c5-4de5-9979-48d014e0a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:05.068905 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-41c549f7-16c5-4de5-9979-48d014e0a68c None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:05.070084 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-41c549f7-16c5-4de5-9979-48d014e0a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:05.078663 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:05.114797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa83f7f-9332-4b80-a95b-c5636d93d37d None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:05.125898 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa83f7f-9332-4b80-a95b-c5636d93d37d None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:05.129685 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ef925-4d20-4b43-b9a4-bc25a64fa14d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:05.130315 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ef925-4d20-4b43-b9a4-bc25a64fa14d service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:05.130605 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ef925-4d20-4b43-b9a4-bc25a64fa14d service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:05.130887 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ef925-4d20-4b43-b9a4-bc25a64fa14d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:05.140350 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ef925-4d20-4b43-b9a4-bc25a64fa14d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:05.141777 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ef925-4d20-4b43-b9a4-bc25a64fa14d service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:05.151094 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 731/1464] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:07.316070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cabbc2-8e9f-4e00-8144-4e71b992f5a3 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:07.316651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cabbc2-8e9f-4e00-8144-4e71b992f5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:07.316651 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cabbc2-8e9f-4e00-8144-4e71b992f5a3 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:07.318078 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:07.323546 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6e06d-3a19-4e10-8458-8588ab42a579 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:07.323988 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6e06d-3a19-4e10-8458-8588ab42a579 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:07.324095 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6e06d-3a19-4e10-8458-8588ab42a579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:07.342293 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-bfc6e06d-3a19-4e10-8458-8588ab42a579 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:07.343488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-bfc6e06d-3a19-4e10-8458-8588ab42a579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:07.351292 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:07.388746 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cabbc2-8e9f-4e00-8144-4e71b992f5a3 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:07.399505 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cabbc2-8e9f-4e00-8144-4e71b992f5a3 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:07.404333 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baad26b-7b79-44f5-9a17-28a42dc6ac1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:07.404960 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baad26b-7b79-44f5-9a17-28a42dc6ac1a service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:07.405244 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baad26b-7b79-44f5-9a17-28a42dc6ac1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:07.405520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baad26b-7b79-44f5-9a17-28a42dc6ac1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:07.414416 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baad26b-7b79-44f5-9a17-28a42dc6ac1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:07.415861 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baad26b-7b79-44f5-9a17-28a42dc6ac1a service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:07.426194 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 735/1467] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:09.603201 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a6b0-3d77-4a66-8013-aad2d8a66a13 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:09.603777 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a6b0-3d77-4a66-8013-aad2d8a66a13 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:09.603777 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614a6b0-3d77-4a66-8013-aad2d8a66a13 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:09.605176 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 15:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:09.610517 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99fead-b2c2-40e4-83f8-3ab19ceaab61 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:09.610969 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99fead-b2c2-40e4-83f8-3ab19ceaab61 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:09.611070 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99fead-b2c2-40e4-83f8-3ab19ceaab61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:09.629502 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-cb99fead-b2c2-40e4-83f8-3ab19ceaab61 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:09.630701 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-cb99fead-b2c2-40e4-83f8-3ab19ceaab61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:09.639300 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 15:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:09.675503 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614a6b0-3d77-4a66-8013-aad2d8a66a13 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:09.687025 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614a6b0-3d77-4a66-8013-aad2d8a66a13 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:09.690512 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cb2f5-8ddd-4f94-9ac6-a27d6f9edf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:09.691137 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cb2f5-8ddd-4f94-9ac6-a27d6f9edf03 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:09.691422 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cb2f5-8ddd-4f94-9ac6-a27d6f9edf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:09.691699 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cb2f5-8ddd-4f94-9ac6-a27d6f9edf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:09.701217 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cb2f5-8ddd-4f94-9ac6-a27d6f9edf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:09.702628 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cb2f5-8ddd-4f94-9ac6-a27d6f9edf03 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:09.711738 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 734/1470] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:19.624140 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=17 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:19.707746 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=17 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:29.829025 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05cc6-0b5a-4620-9df2-fddcde29e251 None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:29.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05cc6-0b5a-4620-9df2-fddcde29e251 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:29.829607 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05cc6-0b5a-4620-9df2-fddcde29e251 None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:29.830980 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 737/1471] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:06:29.836320 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6be76-4863-498a-97d0-a1b6c9dff0c8 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:29.836759 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6be76-4863-498a-97d0-a1b6c9dff0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:29.836901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6be76-4863-498a-97d0-a1b6c9dff0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:29.855817 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-71c6be76-4863-498a-97d0-a1b6c9dff0c8 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:29.857049 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-71c6be76-4863-498a-97d0-a1b6c9dff0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:29.865048 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 735/1472] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:29.912601 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673ccef-1233-4aa3-8b38-34138fb2e81e None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:29.912898 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673ccef-1233-4aa3-8b38-34138fb2e81e None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:29.913189 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673ccef-1233-4aa3-8b38-34138fb2e81e None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:29.914579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 738/1473] 10.222.0.88 () {54 vars in 962 bytes} [Thu Jun 16 15:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:29.920281 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d47f77-09b2-4bda-b331-aee7242b263b None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:29.920585 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d47f77-09b2-4bda-b331-aee7242b263b None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:29.920878 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d47f77-09b2-4bda-b331-aee7242b263b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:29.939764 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-63d47f77-09b2-4bda-b331-aee7242b263b None None] MFA Rules not processed for user `e7ffcec55ac14f5bb8c3af6be2d4c7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:29.941069 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-63d47f77-09b2-4bda-b331-aee7242b263b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:29.947524 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 736/1474] 10.222.0.88 () {58 vars in 1059 bytes} [Thu Jun 16 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:29.954777 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673ccef-1233-4aa3-8b38-34138fb2e81e None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:29.966069 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673ccef-1233-4aa3-8b38-34138fb2e81e None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:29.969915 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cea73-5cf9-4849-9ea9-03d704fae5af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7ffcec55ac14f5bb8c3af6be2d4c7c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:29.970541 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cea73-5cf9-4849-9ea9-03d704fae5af service glance] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:29.970825 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cea73-5cf9-4849-9ea9-03d704fae5af service glance] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:29.971105 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cea73-5cf9-4849-9ea9-03d704fae5af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:29.981038 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072cea73-5cf9-4849-9ea9-03d704fae5af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:29.982459 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072cea73-5cf9-4849-9ea9-03d704fae5af service glance] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:29.991697 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 739/1475] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:32.233262 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68afda-795e-47ad-9f29-96899c2a9bd5 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:32.233884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68afda-795e-47ad-9f29-96899c2a9bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:32.233884 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68afda-795e-47ad-9f29-96899c2a9bd5 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:32.235223 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 737/1476] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:32.240637 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4aacf-3562-46f7-98c0-24ea62f50233 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:32.241114 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4aacf-3562-46f7-98c0-24ea62f50233 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:32.241223 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4aacf-3562-46f7-98c0-24ea62f50233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:32.259682 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-75e4aacf-3562-46f7-98c0-24ea62f50233 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:32.262002 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-75e4aacf-3562-46f7-98c0-24ea62f50233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:32.269681 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 740/1477] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:32.292901 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68afda-795e-47ad-9f29-96899c2a9bd5 None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:32.305206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68afda-795e-47ad-9f29-96899c2a9bd5 None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:32.310498 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a75ec-9fa4-4ec1-87d9-6db2c3c3c6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:32.311136 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a75ec-9fa4-4ec1-87d9-6db2c3c3c6cc admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:32.311423 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a75ec-9fa4-4ec1-87d9-6db2c3c3c6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:32.311717 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a75ec-9fa4-4ec1-87d9-6db2c3c3c6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:32.312464 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a75ec-9fa4-4ec1-87d9-6db2c3c3c6cc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:32.314216 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-388a75ec-9fa4-4ec1-87d9-6db2c3c3c6cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 15:06:32.314216 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: warnings.warn(msg) Jun 16 15:06:32.314216 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: Jun 16 15:06:32.315365 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a75ec-9fa4-4ec1-87d9-6db2c3c3c6cc admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:32.334829 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 738/1478] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:06:32 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:34.550386 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d60fd-151d-4579-9f38-dc54f3f88c7f None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.550903 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d60fd-151d-4579-9f38-dc54f3f88c7f None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.551088 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d60fd-151d-4579-9f38-dc54f3f88c7f None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.552462 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 741/1479] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:34.558461 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d1b89-e17e-4d49-b308-3c27eeb16243 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.562035 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d1b89-e17e-4d49-b308-3c27eeb16243 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.562035 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d1b89-e17e-4d49-b308-3c27eeb16243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.577615 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-e25d1b89-e17e-4d49-b308-3c27eeb16243 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:34.579239 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-e25d1b89-e17e-4d49-b308-3c27eeb16243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:34.587043 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 739/1480] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:34.631512 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d60fd-151d-4579-9f38-dc54f3f88c7f None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:34.645975 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d60fd-151d-4579-9f38-dc54f3f88c7f None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:34.651704 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dfc7f-f75a-4893-b092-7cfec4c63b4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5368284e42d84e77a6fdb88c2e8dcd3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:34.652374 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dfc7f-f75a-4893-b092-7cfec4c63b4a service nova] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.652670 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dfc7f-f75a-4893-b092-7cfec4c63b4a service nova] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.652983 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dfc7f-f75a-4893-b092-7cfec4c63b4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.661396 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dfc7f-f75a-4893-b092-7cfec4c63b4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:34.668556 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dfc7f-f75a-4893-b092-7cfec4c63b4a service nova] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:34.673310 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 742/1481] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:36.993086 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9f43-de83-4f9e-81ff-6eb2658409bb None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.993660 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9f43-de83-4f9e-81ff-6eb2658409bb None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.993660 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9f43-de83-4f9e-81ff-6eb2658409bb None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.995869 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 740/1482] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:37.000943 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb685a-bee1-40e5-aa19-f18b07232765 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.001389 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb685a-bee1-40e5-aa19-f18b07232765 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.001520 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb685a-bee1-40e5-aa19-f18b07232765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.019224 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-8aeb685a-bee1-40e5-aa19-f18b07232765 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:37.020445 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-8aeb685a-bee1-40e5-aa19-f18b07232765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:37.028500 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 743/1483] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:37.064916 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9f43-de83-4f9e-81ff-6eb2658409bb None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:37.075704 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9f43-de83-4f9e-81ff-6eb2658409bb None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:37.079219 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff5d9a-b218-4961-b93c-d37d20dd0337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:37.079858 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff5d9a-b218-4961-b93c-d37d20dd0337 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.080149 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff5d9a-b218-4961-b93c-d37d20dd0337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.080464 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff5d9a-b218-4961-b93c-d37d20dd0337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.089377 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff5d9a-b218-4961-b93c-d37d20dd0337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:37.090798 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff5d9a-b218-4961-b93c-d37d20dd0337 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:37.100065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 741/1484] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:39.278450 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3a30-ee35-493c-9476-0e09d85e79ec None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:39.279057 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3a30-ee35-493c-9476-0e09d85e79ec None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:39.279166 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3a30-ee35-493c-9476-0e09d85e79ec None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:39.280592 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 744/1485] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:39.285896 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420e20e-d420-41f5-bd30-8bed48000835 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:39.286343 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420e20e-d420-41f5-bd30-8bed48000835 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:39.286451 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420e20e-d420-41f5-bd30-8bed48000835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:39.304745 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-3420e20e-d420-41f5-bd30-8bed48000835 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:39.305941 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-3420e20e-d420-41f5-bd30-8bed48000835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:39.314268 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 742/1486] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:39.351570 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3a30-ee35-493c-9476-0e09d85e79ec None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:39.362792 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3a30-ee35-493c-9476-0e09d85e79ec None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:39.366401 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17f384-4082-4947-84ef-5bbe3ed5dc16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:39.367034 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17f384-4082-4947-84ef-5bbe3ed5dc16 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:39.367321 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17f384-4082-4947-84ef-5bbe3ed5dc16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:39.367629 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17f384-4082-4947-84ef-5bbe3ed5dc16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:39.377830 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17f384-4082-4947-84ef-5bbe3ed5dc16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:39.379266 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17f384-4082-4947-84ef-5bbe3ed5dc16 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:39.389129 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 745/1487] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:41.643567 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd512a8c-79d8-4c38-bf90-b2ae5500588a None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:41.644168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd512a8c-79d8-4c38-bf90-b2ae5500588a None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:41.644168 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd512a8c-79d8-4c38-bf90-b2ae5500588a None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:41.645522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 743/1488] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:41.651386 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a62847-fe90-490c-a42f-d02e3a6c8ab7 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:41.651833 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a62847-fe90-490c-a42f-d02e3a6c8ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:41.652165 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a62847-fe90-490c-a42f-d02e3a6c8ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:41.669537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-23a62847-fe90-490c-a42f-d02e3a6c8ab7 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:41.670733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-23a62847-fe90-490c-a42f-d02e3a6c8ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:41.678461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 746/1489] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:41.720729 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd512a8c-79d8-4c38-bf90-b2ae5500588a None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:41.731442 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd512a8c-79d8-4c38-bf90-b2ae5500588a None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:41.734925 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d3c51-77d3-4284-980f-5ccf68200ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:41.735557 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3c51-77d3-4284-980f-5ccf68200ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:41.735849 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3c51-77d3-4284-980f-5ccf68200ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:41.736130 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d3c51-77d3-4284-980f-5ccf68200ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:41.745754 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d3c51-77d3-4284-980f-5ccf68200ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:41.747185 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d3c51-77d3-4284-980f-5ccf68200ff2 service neutron] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:41.756794 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 744/1490] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:44.315420 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3777a5-c0e5-4e80-b920-865dea237beb None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.315995 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3777a5-c0e5-4e80-b920-865dea237beb None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.315995 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3777a5-c0e5-4e80-b920-865dea237beb None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.317449 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 747/1491] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:06:44.323280 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdae92c-ff57-4d90-a20c-c93aa415f098 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.323703 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdae92c-ff57-4d90-a20c-c93aa415f098 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.323949 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdae92c-ff57-4d90-a20c-c93aa415f098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.341416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ffdae92c-ff57-4d90-a20c-c93aa415f098 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:44.342625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ffdae92c-ff57-4d90-a20c-c93aa415f098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:44.350660 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 745/1492] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:44.390576 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3777a5-c0e5-4e80-b920-865dea237beb None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:44.401718 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3777a5-c0e5-4e80-b920-865dea237beb None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:44.405205 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56506585-b44a-4389-b1b8-51ab7d804646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:44.405837 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56506585-b44a-4389-b1b8-51ab7d804646 service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.406176 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56506585-b44a-4389-b1b8-51ab7d804646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.406501 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56506585-b44a-4389-b1b8-51ab7d804646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.418117 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56506585-b44a-4389-b1b8-51ab7d804646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:44.418117 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56506585-b44a-4389-b1b8-51ab7d804646 service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:44.426980 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 748/1493] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:54.336197 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=33 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:54.422853 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=33 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:08:18.731206 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad8c73-1338-461d-a5ff-d916d32b846e None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:18.731803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad8c73-1338-461d-a5ff-d916d32b846e None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:18.731803 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad8c73-1338-461d-a5ff-d916d32b846e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:18.750522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-09ad8c73-1338-461d-a5ff-d916d32b846e None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:08:18.751721 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-09ad8c73-1338-461d-a5ff-d916d32b846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:08:18.760312 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 746/1494] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 15:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:08:18.769623 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d29d4-66aa-4949-840b-8884cb6de254 None None] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:18.770115 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d29d4-66aa-4949-840b-8884cb6de254 None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:18.770115 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d29d4-66aa-4949-840b-8884cb6de254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:18.788425 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.auth.core [None req-ab3d29d4-66aa-4949-840b-8884cb6de254 None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80419) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:08:18.789612 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-ab3d29d4-66aa-4949-840b-8884cb6de254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:08:18.797412 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 749/1495] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 15:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:08:18.801865 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad8c73-1338-461d-a5ff-d916d32b846e None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:18.811996 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad8c73-1338-461d-a5ff-d916d32b846e None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:18.816625 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:18.817275 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:18.817567 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:18.818492 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] PATH_INFO: `/v3/domains` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:18.819224 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:08:18.819535 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:18.820593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 15:08:18.820593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: warnings.warn(msg) Jun 16 15:08:18.820593 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: Jun 16 15:08:18.821472 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:18.832848 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 747/1496] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 15:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:08:18.841172 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d29d4-66aa-4949-840b-8884cb6de254 None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:18.851095 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d29d4-66aa-4949-840b-8884cb6de254 None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:18.855600 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb586fe1-f1d4-4a9b-97c9-a49795816c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:18.856271 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb586fe1-f1d4-4a9b-97c9-a49795816c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:18.856557 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb586fe1-f1d4-4a9b-97c9-a49795816c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:18.856832 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb586fe1-f1d4-4a9b-97c9-a49795816c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:18.857519 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb586fe1-f1d4-4a9b-97c9-a49795816c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:18.859298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-eb586fe1-f1d4-4a9b-97c9-a49795816c29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 15:08:18.859298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: warnings.warn(msg) Jun 16 15:08:18.859298 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: Jun 16 15:08:18.860537 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb586fe1-f1d4-4a9b-97c9-a49795816c29 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:18.968444 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 750/1497] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 15:08:18 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:08:18.972894 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:18.983728 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1de124-9ea9-47fe-a7c2-2414e8d9073e admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:19.040844 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac5678-434e-470b-8bce-1cec29bb2c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:19.041522 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac5678-434e-470b-8bce-1cec29bb2c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.041810 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac5678-434e-470b-8bce-1cec29bb2c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.042090 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac5678-434e-470b-8bce-1cec29bb2c97 admin admin] PATH_INFO: `/v3/users` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.042778 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac5678-434e-470b-8bce-1cec29bb2c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:19.043792 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-9eac5678-434e-470b-8bce-1cec29bb2c97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 15:08:19.043792 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: warnings.warn(msg) Jun 16 15:08:19.043792 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: Jun 16 15:08:19.045027 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac5678-434e-470b-8bce-1cec29bb2c97 admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:19.132654 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 748/1498] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 15:08:18 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:08:19.136885 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb586fe1-f1d4-4a9b-97c9-a49795816c29 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:19.147242 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb586fe1-f1d4-4a9b-97c9-a49795816c29 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:19.151723 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838369b-698f-4ff4-8b62-43823b71d042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:19.152415 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838369b-698f-4ff4-8b62-43823b71d042 admin admin] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.152708 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838369b-698f-4ff4-8b62-43823b71d042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.152991 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838369b-698f-4ff4-8b62-43823b71d042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.153726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838369b-698f-4ff4-8b62-43823b71d042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:19.154723 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-3838369b-698f-4ff4-8b62-43823b71d042 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 15:08:19.154723 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: warnings.warn(msg) Jun 16 15:08:19.154723 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: Jun 16 15:08:19.155638 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838369b-698f-4ff4-8b62-43823b71d042 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:19.169405 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 751/1499] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 15:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:08:19.173473 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac5678-434e-470b-8bce-1cec29bb2c97 admin admin] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:19.184283 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac5678-434e-470b-8bce-1cec29bb2c97 admin admin] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:19.189051 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991fef6-2ffe-45ce-a3b3-b24c5797842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:19.189711 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991fef6-2ffe-45ce-a3b3-b24c5797842f admin admin] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.190038 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991fef6-2ffe-45ce-a3b3-b24c5797842f admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.190325 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991fef6-2ffe-45ce-a3b3-b24c5797842f admin admin] PATH_INFO: `/v3/roles` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.191055 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991fef6-2ffe-45ce-a3b3-b24c5797842f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:19.192065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-d991fef6-2ffe-45ce-a3b3-b24c5797842f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 15:08:19.192065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: warnings.warn(msg) Jun 16 15:08:19.192065 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: Jun 16 15:08:19.193145 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991fef6-2ffe-45ce-a3b3-b24c5797842f admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:19.206518 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 749/1500] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 15:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:08:19.211159 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838369b-698f-4ff4-8b62-43823b71d042 admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:19.221169 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838369b-698f-4ff4-8b62-43823b71d042 admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:19.226579 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d664ac-d3de-4601-a70c-93df1f260595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:19.227491 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d664ac-d3de-4601-a70c-93df1f260595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.227822 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d664ac-d3de-4601-a70c-93df1f260595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.228107 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d664ac-d3de-4601-a70c-93df1f260595 admin admin] PATH_INFO: `/v3/projects/cad3794a725344c0aac2ecb70e902306/users/82301b005e9445b5b2681291aaea08db/roles/4498e695105347909981a684f613b6a2` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.242873 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d664ac-d3de-4601-a70c-93df1f260595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad3794a725344c0aac2ecb70e902306, user_id=82301b005e9445b5b2681291aaea08db, role_id=4498e695105347909981a684f613b6a2)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:19.243929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-d3d664ac-d3de-4601-a70c-93df1f260595 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 15:08:19.243929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: warnings.warn(msg) Jun 16 15:08:19.243929 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: Jun 16 15:08:19.245932 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d664ac-d3de-4601-a70c-93df1f260595 admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:19.355950 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 752/1501] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 15:08:19 2022] PUT /identity/v3/projects/cad3794a725344c0aac2ecb70e902306/users/82301b005e9445b5b2681291aaea08db/roles/4498e695105347909981a684f613b6a2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:08:19.370158 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68d515-40ce-4da9-a8ec-5012c23d5ece None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.370488 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68d515-40ce-4da9-a8ec-5012c23d5ece None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.370797 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68d515-40ce-4da9-a8ec-5012c23d5ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.398073 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-ad68d515-40ce-4da9-a8ec-5012c23d5ece None None] MFA Rules not processed for user `82301b005e9445b5b2681291aaea08db`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:08:19.399277 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-ad68d515-40ce-4da9-a8ec-5012c23d5ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:08:19.463381 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 750/1502] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 15:08:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:08:19.476769 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ab2af-cc3d-4154-879d-752971ff2b9b None None] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.477068 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ab2af-cc3d-4154-879d-752971ff2b9b None None] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.477382 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ab2af-cc3d-4154-879d-752971ff2b9b None None] PATH_INFO: `/` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.478774 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 753/1503] 10.222.0.88 () {54 vars in 963 bytes} [Thu Jun 16 15:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:08:19.483532 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bda806-ce8d-45a3-950c-1da32ea5e009 None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.483857 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bda806-ce8d-45a3-950c-1da32ea5e009 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.484147 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bda806-ce8d-45a3-950c-1da32ea5e009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.502923 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-b1bda806-ce8d-45a3-950c-1da32ea5e009 None None] MFA Rules not processed for user `b39d53d1def347b6b36067e3e93aeb98`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:08:19.504134 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-b1bda806-ce8d-45a3-950c-1da32ea5e009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:08:19.547946 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:08:19.555814 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 751/1504] 10.222.0.88 () {58 vars in 1060 bytes} [Thu Jun 16 15:08:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:08:19.563600 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ab2af-cc3d-4154-879d-752971ff2b9b None None] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:19.575067 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ab2af-cc3d-4154-879d-752971ff2b9b None None] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:19.578771 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a265f0-676a-4847-b3e8-c861eaabc5da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b39d53d1def347b6b36067e3e93aeb98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:19.579404 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a265f0-676a-4847-b3e8-c861eaabc5da service cinder] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.579693 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a265f0-676a-4847-b3e8-c861eaabc5da service cinder] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.579977 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a265f0-676a-4847-b3e8-c861eaabc5da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.588425 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a265f0-676a-4847-b3e8-c861eaabc5da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:19.589856 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a265f0-676a-4847-b3e8-c861eaabc5da service cinder] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:19.598712 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 754/1505] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:08:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:08:19.641844 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b2eb-e66f-406b-b8f4-4d7b93e61d69 None None] REQUEST_METHOD: `GET` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.648338 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b2eb-e66f-406b-b8f4-4d7b93e61d69 None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.648338 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b2eb-e66f-406b-b8f4-4d7b93e61d69 None None] PATH_INFO: `/` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.648338 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 752/1506] 10.222.0.88 () {52 vars in 829 bytes} [Thu Jun 16 15:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:08:19.651047 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a265f0-676a-4847-b3e8-c861eaabc5da service cinder] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:19.714489 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a265f0-676a-4847-b3e8-c861eaabc5da service cinder] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:19.719086 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:19.719949 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.720258 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.720547 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] PATH_INFO: `/v3/users/82301b005e9445b5b2681291aaea08db` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.721714 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] RBAC: Authorizing `identity:delete_user(user_id=82301b005e9445b5b2681291aaea08db)` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:19.722733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: WARNING py.warnings [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 15:08:19.722733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: warnings.warn(msg) Jun 16 15:08:19.722733 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: Jun 16 15:08:19.723936 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:20.014550 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82301b005e9445b5b2681291aaea08db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:20.015200 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82301b005e9445b5b2681291aaea08db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:20.015805 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82301b005e9445b5b2681291aaea08db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:20.027964 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82301b005e9445b5b2681291aaea08db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:20.126597 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.notifications [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82301b005e9445b5b2681291aaea08db'} {{(pid=80419) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:20.144281 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:08:20.152387 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 755/1507] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 15:08:19 2022] DELETE /identity/v3/users/82301b005e9445b5b2681291aaea08db => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:08:20.156911 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14107c5d-9b83-4917-b0f3-ebf1732c0e0f None None] REQUEST_METHOD: `POST` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:20.157406 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14107c5d-9b83-4917-b0f3-ebf1732c0e0f None None] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:20.157513 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14107c5d-9b83-4917-b0f3-ebf1732c0e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:20.176528 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.auth.core [None req-14107c5d-9b83-4917-b0f3-ebf1732c0e0f None None] MFA Rules not processed for user `b841e83eb2b74ca1b301a4644bbdbe93`. Rule list: `[]` (Enabled: `True`). {{(pid=80420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:08:20.178739 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-14107c5d-9b83-4917-b0f3-ebf1732c0e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:08:20.236588 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 753/1508] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 15:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:08:20.246876 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] Authenticating user token {{(pid=80419) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:20.248658 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.fernet_utils [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80419) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:08:20.306610 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c2cc5-ad1f-408c-9160-c76a48ce624b admin admin] Validating token access rules against request {{(pid=80419) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:20.310197 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511e9c4-014a-486c-9f71-a852c915c94b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58a2b345fe914d2cb97756e6fdf30010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7571fbc040844bc29d293fde55eb53ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80419) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:20.310839 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511e9c4-014a-486c-9f71-a852c915c94b service neutron] REQUEST_METHOD: `GET` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:20.311132 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511e9c4-014a-486c-9f71-a852c915c94b service neutron] SCRIPT_NAME: `/identity` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:20.311417 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511e9c4-014a-486c-9f71-a852c915c94b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80419) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:20.321017 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511e9c4-014a-486c-9f71-a852c915c94b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:20.322461 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511e9c4-014a-486c-9f71-a852c915c94b service neutron] RBAC: Authorization granted {{(pid=80419) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:20.331726 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: [pid: 80419|app: 0|req: 756/1509] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 15:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:08:20.889196 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14107c5d-9b83-4917-b0f3-ebf1732c0e0f None None] Authenticating user token {{(pid=80420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:20.890687 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.fernet_utils [None req-14107c5d-9b83-4917-b0f3-ebf1732c0e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:08:20.903070 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14107c5d-9b83-4917-b0f3-ebf1732c0e0f None None] Validating token access rules against request {{(pid=80420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:20.907731 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b841e83eb2b74ca1b301a4644bbdbe93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d718157b3b949fc80592326cc9cfb5a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:20.908620 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:20.908907 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:20.909185 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] PATH_INFO: `/v3/projects/cad3794a725344c0aac2ecb70e902306` {{(pid=80420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:20.910416 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad3794a725344c0aac2ecb70e902306)` {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:20.911433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: WARNING py.warnings [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 15:08:20.911433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: warnings.warn(msg) Jun 16 15:08:20.911433 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: Jun 16 15:08:20.913339 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] RBAC: Authorization granted {{(pid=80420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:21.131828 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad3794a725344c0aac2ecb70e902306', 'request_id': 'req-873ddc3e-8f84-47d8-93fe-642449d172ff'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:21.137727 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad3794a725344c0aac2ecb70e902306', 'request_id': 'req-873ddc3e-8f84-47d8-93fe-642449d172ff'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:21.194476 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad3794a725344c0aac2ecb70e902306', 'request_id': 'req-873ddc3e-8f84-47d8-93fe-642449d172ff'} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:21.213830 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] The token cache is being invalidate because project cad3794a725344c0aac2ecb70e902306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:08:21.214119 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:21.214867 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG keystone.notifications [None req-873ddc3e-8f84-47d8-93fe-642449d172ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:08:21.216441 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: [pid: 80420|app: 0|req: 754/1510] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 15:08:20 2022] DELETE /identity/v3/projects/cad3794a725344c0aac2ecb70e902306 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:08:30.327586 dv-cinder-837353-2-iscsi devstack@keystone.service[80419]: DEBUG dbcounter [-] [80419] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80419) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:08:31.210929 dv-cinder-837353-2-iscsi devstack@keystone.service[80420]: DEBUG dbcounter [-] [80420] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}